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Because of the duration and nagnitude of this OYerai 
project. Maintaining inter-rater reliability presented seweral 1 
problens. Since the study vas designed to conpare teacher behawior I 
across a three*year tine span, it vas necessary to aaintain \ 
reliability across that tine period. The use of student raters posed^ 
additional problens of nobility and lack of professional connitnent.l 
In order to reach and naintain the desired leYels of reliability, ^ 
solutions to these problens were deYised in two najor areas: (1| 
coanitnent building«*activities designed to sake the raters beliewe 
that the work they were doing was worthwhile and inportant; and (2): 
training*activiti.es designed to help students reach the desired lewell 
of reliability on ratings. The najor portion of the docunent contains- 
the actual training nodules used in the rater training actiwities. 
(Author/PC) 



MAINTAINING RELIABILITY 

in a 

LONGITUDINAL STUDY 



inttrim Rtport No. 1 
NatkNMi Institute of HMlth 
Grant No. 5 PO 1 MH 19871 



by 

F. N. Rotbuck 
D.N.Aspy 
LL. Sadler 
M.A.Willson 



"^eWIISSIOM TO REMOOUCE THIS COPY- 
nCHTED MATEDUU. HAS KEN GRANTB) tY 



U S. OC^MTMCMT OF HCALTM. 
EOUCATIOM a«lCL»*C 
NATIOMAL IMtTITUTE Of 
EDUCATIOM 



This document has seen repro 

DUCEO exactly as RECEIVED FROM 
The person or ORCANtZATtCN origin 
ATlNCIT POINTS 0' VfEMTO' OPINIONS 
STATED DO NOT NECESSAl ILV REPRE 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION POSITION OR ^OLtCV 



TO ERC AND ORGAMZAnONS OPERATING 
(MOCR AGREEMENTS WITH THE NATIONAL IN- 
STITUTE OF EDUCATION FURTHER REPRO- 

oucnoN ouTSloe the emc system re- 

O-JIRES PERMISSION OF THE COPyRK;HT 
OMNER- 



NATIONAL CONSORTIUM FOR HUMANIZING EDUCATION 

SKILLS TRAINING FOB HUMAN GROWTH 



MAINTAINING RELIABILITY IN A LONGITUDINAL STUDY 

by 

Flora N. Roebuck 
David N. Aspy 
Lynda L. Sadler 
Martha A. Willson 



Interim Report Number 1 
of the 

NATIONAL CONSORTIUM FOR HUMANIZING EDUCATION 
Northeast Louisiana University 
Monroe, Louisiana 
1974 



The research reported here was supported in 
part by NIH Research Grant #P0 1 MH 1987I 
from the National Institute of Mental Health 



I f 



COPYRIGHT 1974 BY F. N. ROEBUCK 



ERIC 



i 



ACKNOWLEDGMENTS 



The Principal Investigator and the Executive Director of the 
National Consortium for Humanizing Education want to take this 
opportunity to acknowledge their gratitude and appreciation to 
all the student workers. The students' commitment to the 
research project went far beyond that of employee to employing 
organization. Their willingness and high quality performance 
contributed greatly to the success of the project. A sincere 
"Thank You" is extended to each one of them. 

A special debt of gratitude is owed to those students 
who "went the extra mile" and worked on holidays or during 
between-semester breaks, initiated helpful or time and effort- 
saving routines, undertook every task with a "let me at it" 
attitude, and were responsive to the needs of their co-workers. 
It is our pleasure to acknowledge the debt we owe — the 
research couldn't have been done without them: 



Debbie D. Brown 
Kay Gilliland 
Terri Grantham 
Betty Herrington 
Doug Holmes 
Ruby James 
Kathryn Jordan 



Lisa Kicker 
Virginia Lindow 
Debbie '^McBeth 
Dale Mahaffey 
Pat Redd 
Barbara Roberds 
Jo Ann Robinson 



s 



TABLE OF CONTENTS 

Page 



OVERVIEW ' , _ ^ ^ 

Commitment Building Activities . . . . 2 

Rater Training Activities 4 

Reliability Checks. . 7 

TRAINING MODULES . ' 

Interchangeable Response 12 

Flanders* Interaction Analysis 13 

Cognitive Functioning Categories 22 " 

Introduction, to Process Scales and 

Measuring Energy level. . * . 31 

A Scale for the Measurement of a Teacher's 
z ^ Understanding of the Meaning of Classroom 

Experiences for her Students 36' 

A Scale for the Measurement .of a Teacher's 
Genuineness in her Classroom Interaction 

with Students . 42 * 

A Scale for the Measurement of the Process 

of Success Promotion 47 

A Scale for the Measurement of Respect Provided 

by the Teacher in Classroom Interaction 53 

A Scale for the Measurement of Student 

Involvement ' 59 

APPENDIXES 

A - Illustration of Categories A-1 - 11 

B - Master Ratings for Training Tapes B-1 - 18 

- , * 

C - Location of Tapes . , C-1 



ERIC 



MAINTAINING RELIABILITY IN A LONGITUDINAL STUDY 



Because of the duration and magnitude of this project, 
maintaining inter-rater reliability presented several problems. 
Since the study was designed to compare teacher behavior across 
a 3-year time-span, reliability had to be maintained across that 
same period. That is^ if a particular kind of teacher behavior 
was rated "2*5" in September, 1971, then an occurrence of that 
same kind of behavior in April, 197^5 must also be rated "2,5." 

Because of the magnitude of the project, with several 
thousand hours of instruction to be rated each year, a large 
crew of raters was necessary. The only appropriate and readily- 
available labor pool from which to draw these raters was the 
students at Northeast Louisiana University, However, the use 
of student labor posed additional problems: 

1. Mobility — Students graduate, leave school, get married, 
no longer need to work, or find a different Job. Change 
in rater-crew membership from semester to semester was 
approximately 30%. Only one member of the rating crew 
in the Spring of 1974 had also been a member in the 
Pall of 1971. 

2. Lack of Professional Commitment —To the students, 
rating was "just a Job" — they had no investment in 
high quality accuracy. Furthermore, after the first 
fifty or so tapes, listening to them became a very 
monotonous activity* We planned to rate 6,000 tapes. 

In order to reach and maintain the desired levels of 
reliability, solutions to these problems were devised in two 
major areas: (1) Commitment Building and (2) Training. The 
success of these efforts were constantly evaluated through 
Reliability Checks and the results of the checks were fed back 
to the raters as individual reliability scores • Raters who fell 
below standards were recycled through training before being 
allowed to rate again. 

Commitment Building activities were designed to elicit 
within the raters the beliefs that (1) the work they were doing 
was worthwhile and important and (2) they were "special" because 
they had earned the right to do this work. The activities 
carried out included: 
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1. Selection Procedures 

2. Publicity 

- 3. Usage Bulletin Board 

4. Field Trips 

5. Physical Fitness Programs 

6. Counseling Services 

Rater Training af^tivitles were standardized so that a new 
rater cror example, trained in the Fall of 1973) received the 
same training as earlier trainees. The training activities were 
of 3 kinds:. 

Initial Training 

2. Continuing Training 

3. Recycling for falling Reliability 

Reliability Checks were carried out periodically in both 
inter- and intra-rater reliability areas. These checks included: 

1. Weekly Inter-rater Reliability 

2. Monthly Transference Spot-checks 

3. Longitudinal rate-rerate checks across rater crews 

Each of the activities listed above are described more fully 
below. 



Commitment Building Activities 

Commitment building activities were initiated in January, 
1972, because transference checks (described below) during the 
Fall of 1971 had revealed the fact that some of the raters were 
not transferring their skills to the actual research rating 
situation. Each of the specific procedures carried out in 
tuilding rater commitment is explained below. 

1. Selection Procedures 

The students were Interviewed and evaluated according to 
their energy level, posture, eye contact, expressed interest in 
the research program, and attitude toward learning. Students 
were sought who liked physical exercise, who had a moderate 
involvement in extra-curricular activities, and who maintained 
a 2.0 (or above) grade point average. It was most important that 
the student be able to do detailed repetitive work. The members 
of the rating corps were selected so as to always contain both 
males and females and at least a black/white ratio. 
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Each applicant agreed to a 4-hour trial training period 
without pay. At the end of this time, those students who (1) 
liked the work and (2) had demonstrated that they were capable 
of doing it accurately were retained as trainee-raters. These 
trainees then completed the Initial Training Phase (described 
below)* As an incentive, they were paid for initial training 
only upon successful completion of the course. 

Trainees did not rate .research tapes until they could 
maintain 90% (p<.01) inter-rater reliability. When a rater's 
production reached the desired level of 2h-3 tapes rated per 
hour with minimum error, the rater received a pay increase. 

2. Publicity 

Once the raters to be hired had been selected, press releases 
were sent to their hometown newspapers . Stories and pictures 
about the Consortium and its raters were placed in the student 
and local newspapers, and a paragraph and a picture of the 
current' raters was used in the NLU Bulletin: General Information 
Issue, a recruitment booklet sent by the University to high 
school students all over the state. Copies were provied for 
each rater, as well. 

3. Usage Bulletin Board 

A bulletin board in the rater sign-in area was posted with 
items designed to emphasize the importance of their work. It 
was kept current with newspaper clippings of stories about the 
Consortium and with letters requesting research results and- other 
information about the Consortium's work which had been received 
from all over the United States and several foreign countries. . 
Additionally, whenever one of the raters received an honor or 
award, a newspaper clipping or other notice of his achievement 
was posted here. 

i|. Field Trips 

From time to time, one or more raters were selected to 
accompany staff members on trips to school systems (other than 
the experimental system) which had requested presentations and/or 
training in the Consortium technology. During these trips, the 
raters received firsthand evidence of the importance of the 
Consortium's work as well as the experience of teaching others 
about what they were doing. 
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5. Physical Training 



It soon became apparent that the more physically fit a ratf^r 
was, the better his reliabili-.y It is hypothesized that this is 
a result of a higher energy level enabling more concentrated 
attention to the task at hand. At any rate, we initiated a program 
of encouraging each rater to develop a physical fitness training 
program for himself. Each rater's progress in his program was 
charted on the bulletin board. To Increase participation, a rater 
who earned 100 Aerobics points received a NCHE sweatshirt with the 
motto, "I ratel", 

6. Counseling 

Counseling was not a pre-planned part of the program but 
evolved as the students discovered the office climate of 
genuineness and positive regard. Raters would periodically seek 
out .private conference time for their personal, .financial, and 
school-related problems. The Consortium was where they could 
be themselves and find their own solution to problems. 

The pace of work was very hectic at times, but the students 
always completed the necessary tasks with a sense of togetherness 
in accomplishment. The Consortium was "theirs."' 



Rater Training Activities 

Rater training was standardized in three ways: 1) the 
training was conducted with standardized modules, 2) the modules 
were presented in a standardized sequence, 3) initial training 
practice in rating utilized standardized training tapes with 
master ratings. Before completing training, the rater trainees 
were expected to be able to reach .95 reliability with the master 
ratings. 

Training activities were conducted in three phases: 1) 
initial training, 2) continuing training, 3) recycling. Each of 
these is explained more fully below, 

1. Initial Training 

During the initial training phase the rater trainees were 
presented with a series of standardized modules in a standardized 
sequence. The modules presented to the raters and the master 
ratings for the standardized training tape are included in this 
report. Copies of the training tapes have been submitted to the 
National Institute of Mental Health along with this report. 
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Following the presentation of each of the measurement 
modules, the raters practiced using the instruments singly and 
(as the training process continued) in combination with other 
instruments. The sequence of presentation of the modules and 
the associated rating practice" periods is presented in the 
generalized training schedule shown in Table I, page 6. The 
training sequence remained the same from semester to semester; 
however, the t^me involved in the rating practice periods varied 
slightly with the abilities of the various rater crews. 

2. Continuing Training 

As rating teams reached the desired level of reliability, 
they were released from the initial training phase and began 
actual rating work. When all of the teams had completed the 
initial training phase, the continuing training phase began. 
In this phase, a Rater Meeting was held once a week for approxi- 
mately ah hour and a half. 

These weekly meetings were mandatory for all raters. The 
meetings began with matters of business affecting the rating 
crews; such as, arranging rating schedules for teams, changes in 
rating procedures and/or team assignments, and the correction 
of errors in rating tabulation. This usually occupied only a 
few minutes of the meeting. The two major activities for the 
rater meetings were: (1) continuing training for rating accuracy, 
(2) the reliability checks (described below). 

As a standard procedure, rat;ers were instructed to set 
aside tapes they came across (while rating) which contained either 
unique examples or very good examples at the high or low 
extremes of the instruments. The rater supervisor then made 
selections of two or three segments from those tapes to be 
used in the Continu,-lng Training for rater accuracy. At the rater 
meetings, these tapes were presented for rating. 

The ratings were done individually by each rater. Each 
segment was rated on all seven instruments. Then the ratings 
for each segment were revealed by each rater. Whenever disagree- 
ment occur ed, the segment was replayed and discussed until a 
consensus was reached as to the proper rating for the tape. It 
was in these discussions that the raters internalized the scales 
and mastered the fine points of rating. 

3. Recycling 

If a rater fell below the desired reliability in the weekly 
reliability checks (described below), he was removed from his 
rating team and recycled through the training. In individual 
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TABLE It GENERALIZED RATER TRAINING SCHEDULE 
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Introduction to Consortium 
Interchangeable Response Ivicdule 

Practice on Interchangeable Response 
Flanders Interaction Analysis Kodule 
Practice with Flanders Interaction Analysis 

Practice on Flanders* 

Introduction to Cognitive Functioning 
Interchangeable Responses 

Cognitive Functioning Categories Kodule 
Interchangeable Responses 

Flanders Practice 

Practice with Cognitive Categories 
Interchangeable Responses 

Cognitive Practice 

Energy Level I^odule (Introduction to Process 
Scales) 

Interchangeable Responses t Test of Accuracy 
and Communication 

Review Energy Level 

Meaning Process Scale (Kodule and Practice) 
Interchangeable Responses Test 

Genuineness Process Scale (Module and Practice) 
Success Promotion Scale (Module and Practice) 
Practice with Flanders and Cognitive Categories 

Practice with 3 Process Scales combined 
Respect Process Scale (Module and Practice) 
Student Tr/^ulvement Process Scale (I/iodule u 
Practice) 

Practice with all 5 Process Scales 
Practice with Flanders and Cognitive 
Practice with all 7 instruments 

Practice with all 7 instruments 
Assignment in Pairs* Pairs practice with all 
7 instruments* 



Assignment to Teams. 
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sessions with the rater supervisor, the module (s) for the 
particular instrument (s) in which the rater ^s reliability had 
slipped were re-presented. In addition, the rater was assigned 
additional practice rating with a Senior Rater. A Senior Rater 
was a. rater who had been with the Consortium for more than one 
continuous semester and who had maintained high rater reliability 
throughout his period of employment. 

In the practice rating, selected tapes were rated by both 
the Senior Rater and the rater receiving recycling. At the end 
of each segment, both ratings were compared. Wherever there 
was a discrepancy, the Senior Rater explained his ratings. This 
was continued using different tapes until the desired level of 
agreement between the two ratings was reached. 

At this point in time, the rater receiving recycling then 
rerated the master training tape for his area(s) of weakness. If 
he received .95 reliability with the master ratings, he was 
returned to the rating team; otherwise training would be continued. 
During the life of the project, no situation arose in which a 
second cycle of retraining was necessary. 



. Reliability Checks 

Three kinds of reliability checks were conducted. Inter- 
rater reliability was checked at each weekly rater meeting. 
In order to make sure that the rater teams were transferring 
their reliability to the actual research rating when working at 
the rating stations, spot checks of the rating teams were 
carried out once a month by Senior Raters. Beginning with the 
second year, inter-crew longitudinal reliability checks were 
conducted once a semester. 

1. Weekly Inter-rater Reliability Checks' 

The weekly inter-rater reliability checks were handled in 
the weekly rater meetings. Two of the standardized training 
segments were presented for ratings. In order to ensure that 
reliability was not an effect of remembered ratings on the 
standardized tapes (even though each segment was used only 
twice in 25 weeks), a third tape segment was pre-selected and 
standardized by the rater supervisor for each meeting. "Master" 
ratings for this third segment were established by a consensus 
rating of the rater supervisor and three of the Senior Raters. 

Manual rating sheets rather than the computer rating sheets 
were used in this check. The manual rating sheets included 
places for each recorded response and spaces for category 
tabulation. 
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The raters were divided into two groups. On the first 
segment one half rated Flanders while the other half rated 
Cognitive. All rated all of the process scales. On the 
second segment, the assignments to Flanders and Cognitive rating 
were reversed. 

Each rater rated individually^ At the end of the segment, 
each rater tabulated his Flanders and/or Cognitive ratings by 
category to obtain totals for each category. 

The rating sheets were collected by the rater supervisor 
for use in checking inter-rater reliability. Additionally, 
results of the ratings were charted and presented to the raters 
at the next meeting. This gave a quick feedback to the raters 
on their individual agreement with the others. A sample chart 
is shown on page 9. 

The cross-hatched area of the chart represents the maximum 
extent of ,90 reliability. If a rater was outside this range, 
he was below standard, (See Rater #15 in illustration on p. 9) 

The allowable range of agreement (at r = .90) for the 
process scales was a .difference of not more than ± .3 from the 
master ratings. For the totals of the Flanders and Cognitive 
Categories (except those noted below), the raters were expected 
to agree with the master codings to a difference of no mox^e 
than i 4 tallies. For some of the *Flanders and Cognitive 
categories which occur relatively infrequently but which were 
believed to be very impoi^tant in teacher-pupil interaction, the 
raters were expected to identify and record every occurrence of 
that category.. These categories were Flander's Categories 1, 2, 
and 3 and Cognitive Categories 3, 4, 7, and 8. 

2. Monthly Spot Checks of Transference 

The purpose of this spot check was to verify that the rater 
teams transferred their rating skills from weekly reliability 
sessions to actual rating time. The procedure was simply to 
rerate samples from the tapes rated each month. 

Two senior Raters selected at random a tape rated by each 
team during the preceeding month. At the original rating of 
the tape, the raters recorded the footage number for each segment 
which they rated. In rerating the tape, the Senior Raters were 
therefore able to rate the same segments of the tape as had been 
rated originally. 
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For this check, the Senior Raters used a special score sheet 
on which space was provided for rerating of the tape as well as for 
the original rating. After rerating the tape selected and doing 
the category tabula^on for Flanders and Cognitive, the Senior 
Raters pulled the original ratings from the file and recorded them 
on the check shee^. . 

The comparison of the rerate and the original rating was the 
basis for the- check. These checks were conducted throughout the 
duration of the project. However, after the initiation of Commit- 
ment Building Activities in January, 19?2, no instances were found 
in which the rating skills had not been transferred from the 
reliability meetings to the actual research rating situation. 

3. Longitudinal Inter-Crew Reliability 

Because of the need to be able to compare teacher ratings 
across three years, the most important check carried out was the 
longitudinal rate-rerate across rater crews. The purpose of the 
weekly reliability and the monthly spot checks was to ensure that 
when the longitudinal rate-rerate checks were made, they would be 
within tne desired level of reliability. In conducting the 
mter-crew reliability check each semester, the tapes from one 
month of the year before, which had been rated by the rater crew 
of the preceeding year, were selected and re-rated by the current 
rating crew. The tapes were assigned at random to the current 
rating teams. As the ultimate rate-rerate check, the rati»ig 
crew of the Spring, 197^, rerated tapes from Fall of 1971. 

The Pearson's Correlation Coefficients for this final 
longitudinal rate-rerate across crews are displayed in Table 2 
shown on page 11. • . 

The coefficients for the longitudinal rate-rerate reliability 
across three years revealed a higher variability in the reliability 
or the instruments than had been exhibited in the weekly inter-rater 
reliability checks. The longitudinal coefficients ranged from .898 
to .963 (See Table 2), with category codings for Flanders Interaction 
Analysis proving the most reliable across time. Among the process 
scales, the Student Involvement scale had the highest coefficient 
while tne Respect scale proved the least reliable. However all 
coefficients were significant at p. <.01. * 

In the following section of this report, the rater training 
modules and the materials which accompany each module are presented 
The master ratings for the standardized training tapes are located in 
Appendix B. o f a 
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Table 2: Pearson^s Correlation Coefficients for 

Longitudinal Rate-Rerate Reliability Across Rating 
Crews from Pall of .1971 to Spring of 197^ 



Instrument 
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INTERCHANGEABLE RESPONSES: DISCRIMINATION AND 
COMMUNICATION OP FEELINGS 

. The National Consortium for Humanizing Education received 
special permission from Dr. R. R. Carkh'uff to utilize in rater 
training the module on Interchangeable Responses which he had 
developed from the material in his book The Art of Helping . 
The module is not presented here as Dr. Carkhuff retains copy- 
right to the material. Communications about this module should 
be addressed to Dr. R. R. Carkhuff at Human Resource Development 
Press, P. 0, Box 628, Amherst, MA - 01002. 
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FLANDERS' INTERACTION ANALYSIS 



OBJECTIVES 



1) To establish a trust relationship between the instructor 
and each trainee so that the trainee will feel secure 
enough to examine teaching behavior. 

2) To present Flanders' Interaction Analysis so that each 
trainee comes to see it as one tool with which he can 
examine any teaching behavior. 

3) To present Flanders' Interaction Analysis so that each 
trainee can identify the category and/or categories of 
a given teacher-student interchange. 

4) To develop proficiency in the use of Flanders' scale so 
that each trainee can identify the category of interaction 
at regular intervals. 

5) To refine proficiency so that eacn trainee can code any 
given segment of tape of verbal exchange with Flanders^ 
Interaction Analysis at regular intervals of 3 seconds. 
He will be able to do this with a difference from the 
.trainer's coding of no more than 4 in the total number 

of tallies in each category, and the trainee will identify 
and record every occurrence of Categories 1, 2, and 3. 



MATERIALS 

1) Flanders' Categories for Interaction Analysis (copies 
for each trainee) 

2) Flanders' and Cognitive Training Tape (Appendix C) 

3) Master ratings for training tape (Appendix B) 

4 ) Metronome 

5) Cassette recorder 

6) Chalkboard and chalk 

7 ) Paper 

8) Pencils 

9) Transparencies FT-l, FT-25 FT-S 

10) Overhead projector 

11) Illustration of Categories #1, #2, and #3 

12) Appendix A 
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PROCEDURES 



As introduction, ask questions about the interactions and 
activities that occur in the classroom. Questioning could be 
phrased in this way: "What goes on in a classroom?" (List 
each activity on the board as it is named). "Could we say 

upon examination that most of the activities are verbal? 

that verbally we indicate each activity that is going on?" 
"What are the different types of verbal interchange?" List 
all types on board. Accept all answers. 

This discussion is followed with an explanation by the 
instructor of Flanders' Categories, Use the transparencies 
at this point.* General points to be made: 

a) Caregories are descriptive, not evaluative. 

b) 97% of everything that goes on in the classroom can be 
determined by verbal behavior, thus Flanders' category 
is effective for use in research. 

Apply Flanders' category numbers to different types of verbal 
behavior listed on board. If some need to be broken down, 
break them down through questioning a sample sequence, such 
as, discussion brought on by asking a question, which initiated 
response, which in turn motivated someone else to respond. 
Three categories (ij, 8, 9) are thus brought forth through " 
closer examination. 

See transparency FT-l: Function of Flanders'. 

This transparency shows the function of the Flanders' categories. 
It describes - who is talking, type of talking going on (refer, 
back to listing of types on board), and what follows what in 
the type of talk (question followed by answer, followed by praise 
etc . ) 

Examine individual copies of Flanders' category with transparency 
of Flanders' Categories FT-2. 

Point out these major divisions of Flanders' Categories: 

a) The category is broken down as to who is talking — teacher, 
student, or no identifiable person (silence or chaos). 



"Masters for the transparencies used with the module follow 
this module. 
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b) Teacher talk is broken down as to whether the teacher 
is using indirect teaching methods (accepts feelings, 
praises, uses student ideas, asks questions) or .whether 
she is using direct teaching methods (lectures, gives 
directions, criticizes). 

5) Read each category aloud. Have a trainee take each category, 
read, explain, and give examples. In the discussion these 
points should be made: 

a) There is a distinction between categories 1 and 3; 
category 1 is accepting student feelings ; category 3 
is accepting or using student ideas . 

b) There is a distinction between categories 8 and 9 — 
category 8 is a teacher elicited response — called for 
directly by the teacher; category 9 is a student 
initiated response — the student wants to respond or to 
add something to the class. 

c) Emphasize that no category is "bad". The good teacher 
uses whatever category is appropriate to the need of the 
situation and to the immediate instructional goals. 
Flexibility is important-- the ability to hit whatever 
category is needed by the class. 

d; Distinguish verbal tics from genuine praise as coded in 
category 2. A verbal tic is a "praise" type word used 
so often it becomes meaningless such as "good", "o.k.", etc. 

6) Divide into small groups and do Illustration of Categories #1;* 
Return to the large group and discuss. Each group should be 
able to give reasons for choosing the category that they did. 
No one is wrong. The instructor simply gives his way of 
"hearing"' the different responses on the worksheet. Explain 
that the basic purpose is for the group to "hear" it alike. 

7) Individually do Illustration of Categories #2.* Discuss as a 
group. Same rules apply as in procedure 6. 

8) Provide a brief success experience with the categories. Have 
the trainees relate their classroom experience to Flanders' 
categories by translating category numbers into familiar 
patterns of interaction. Use the transparency of Flanders' 
Patterns, FT-3. List the different patterns and ask the 
trainees to tell what is going on. 



♦Appendix A 
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PATTERNS 



a 



c 



e 



h 



i 



8 
8 



10 
8 



4 
10 
8 
9 

.9 



5 
5 

U 
8 



5 
5 
6 
8 
8 



10 
7 



0 



5 
5 



5 
5 
5 
9 
5 



8 
2 
9 
9 



8 
2 
2 
3 



Points to be made for each pattern: 

a) ' Question and answer. More likely to be a drill than a 

discussion. 

b) . Teacher asked question: Student unable to answer. 

Teacher rephrased question. Student answers. 

c) Teacher asked question: pause for thinking, students 
answered questions. Presence of 9 indicates the question 
stimulated thinking. The 10 can be indicative of 
thinking time or of confusion* 

d) Lecture, question, answer. Question probably called 
for recall of item in lecture. • 

e) Lecture, directions, student compliance with directions. 

f) Lecture, directions, students failed to comply, teacher 
criticizes students^ failure. 

g) Lecture interrupted by student question. Reply to question. 

h) Question, answer, praise, unsolicited participation. 

i) Question, answer, unsolicited response, praise, clarifying 
and building on idea presented. 

^ 9) Divide into small groups and work out original samples ^ of 
various patterns to be presented and coded by whole group. 
Patterns may be assigned and/or the group may choose the 
pattern they wish to demonstrate. The whole group will code 
the interaction at every exchange or at regular measured 
intervals of 5, ^, or 3 seconds. 

10) At this time it should be brought out that inside of one 

interval several exchanges may occur, or even one person can 
use several categories inside an interval. The coder should 
code the category which is most significant for the interval. 
This could be the category which lasts the longest or the 
category which does not occur often, such as 2 or- 1. The 
important overall view is the pattern the numbers form which 
should be representative of the verbal behavior. 
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11) Listen to one segment of Flanders' training tape. Turn 
the metronome on a slow speed and code the segment once 
every beat. Write code numbers ir> sequence on a sheet of 
paper. Discuss and compare codings by counting categories, 
placement, and patterns. This should be repeated on various 
segments until the desired proficiency is attained. 

12) If necessary in order to reach reliability standards , the 
^ following additional training can be conducted: 

a) Work on "Illustrations of Categories #3'^ Repeat 
procedure 7. 

b) Repeat procedure 11 with other training tapes. 



OUTCOME INDICES 

For objective 1 ; 

Each trainee maintains an alert attentive physical posture 
and participates in the activities of the group. 

For objective 2 ; 

[ Each trainee identifies patterns in a, verbal classroom 
interaction. 

For objective 3 : 

bacn trainee codes the worksheets on an S5% accuracy with 
that of the instructor. 

For objective 4 ; 

sacn trainee codes on a slow speed such as a 5 second 
interval. This coding will compare with that of the 
instructor so that there is no more difference than 4 in 
each category total, and for each category that is more 
than 0 but less than 4, he codes at least 1. 

For objective 5 : 

Each trainee codes any given segment of tape of verbal 
exchange on a three second interval on Flanders'. He does 
this with no more of a difference from the trainer's coding 
than 4 in the total number coded in each category, except 
that the trainee will identify and record every occurrence 
of categories 1, 2, and 3. 
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FLANDERS* 

CATEGORIES FOR INTERACTION ANALYSIS* 



u 

0) 

•g 



0) 
V 

rH 



V 

u 



Accepts Feelings ; Accepts and clarifies the feeling 
tone of the students in a non-threatening manner. 
Feelings may be positive or negative. Predicting or 
recalling feelings are included. 

Praises or Encourag es; Praises. or encourages student 
action or behavior. Jokes that release tension, ret 
at the expense of another individual , nodding head or 
saying, "urn hm?" or "go on" are included. 

Accepts or Uses Ideas of Student : Clarifying, building, 
or developing ideas suggested by a student. As teacher 
brings more of his own ideas into play, shift to 
Category 5. 

Asks Questions ; Asking a question about content or 
procedure » With the intent that a student answer. 



^4 



V 
0) 

u 

P 



Lecturing: Giving facts or opinions about content or 
procedure; expressing his own ideas, asking rhetorical 
questions.^ 

Giving Directions ; Directions, commands, or orders to 
which a student is expected to comply. 

Criticizing or Justifying Authority ; Statements 
intended to change student behavior from non-acceptable 
pattern; bawling someone out; stating why the teacher 
is doing what he is doing; extreme self-reference. 



M 
H 

C 

0 

CO 



student Talk: Response; talk by students in response 
Teacher initiates the contact or solicits 



to teacher, 
student statement. 



Student Talk ; Initiation; talk* by students which they 
initiate. If "calling on" student is only to indicate 
who may talk next, observer must decide whether student 
wanted to talk. If he did, use this category. 



10. Silence or Confusion ; Pauses, short periods of silence 
and periods of confusion, in which communication cannot 
be understood by the observer. 



♦Minnesota, 1959 



J 
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FT- I: function of FLANDERS 

I. DESCRIPTIVE (non - evaluative) 

(a.) WHO IS TALKING? 

(b.) TYPE OF TALK 

(c.) SEQUENCE OF TALK (what 
f 0 I I ows what ) 
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FT-2: F I onders Ana I vs i s 





1 
1 


1 

1 . 




n 




T . 
1 


H 

u 




e 


• 

1 


2 . 


a 


r 




c 


e 


3. 


h 


c 




e 


t 


V. 


r 






T 


n 

u 

• 

1 


5. 


a 




1 


r 


6. 


k 


e 






c 
t 


7. 



Student 
Talk 



foe I i ngs 
2. praises 

uses student ideas 
V. asks questions 



5. lectures 

6. gives directions 

7. criticizes 



8, Student responds 
?. Student initiates 



10. silence or "confusion 
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FT-3: FLANDERS' PATTERNS 
A B C • D E F G H 1 

8 10 10 5 5 5 5 8, 8 

88<7V8I0 '?<?2 
'/V<78875<?3 
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COGNITIVE FUNCTIONING CATEGORIES 



OBJECTIVES 

1) To maintain a trust relationship so that each trainee 
participates in discussion and contributes his own ideas. 

2) To present the Cognitive Categories so that each trainee 
comes to see it as a tool with which he can measure the 
amount of thinking occurring in a classroom. 

3) To present the Cognitive Categories so that each trainee 
can distinguish the memory categories from the thinking 
categories of a given teacher-student interchange, 

4) To develop proficiency in the use of the instrument so 
that each trainee can identify the appropriate Cognitive 
category at regular intervals. 

5) To refine the proficiency so that each trainee can code any 
given segment of tape of verbal interchange with the Cognitive 
Categories at regular intervals of 3 seconds. He will be able 
to do this with a difference from the trainer <s coding of no 
more than ^ in the total number of tallies coded in each 
category, and the trainee will identify and record every 
occurrence of categories 3, 4, and 8. 



MATERIALS 

1) Cognitive Categories (copies for each trainee) 

2) Flanders » and Cognitive training tape (Appendix C) 

3) Master ratings for training tape (Appendix B) 
^) Metronome 

5) Cassette recorder 

6) Chalkboard and chalk 

7) Paper 

8) Pencils 

9) Transparencies CT-1, CT-2, CT-3 

10) Overhead Projector 

11) Illustration of Categories #1, #2 and #3 (Appendix A) 
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PROCEDURES 

Show transparency CT-1 of COGNITIVE MAJOR CONCEPTS. Read 
it aloud, discuss., and call for questions. 

Major Concepts: 

a) Cognitive means knowledge. . .this scale is. concerned 
with the knowledge or learning that is taking place. 

b) This instrument focuses strictly on the Intellectual 
sphere of activity . We are interested only in the mental 
processes occurring in the classroom. All emotional 
things are put in one category — 9- 

c) Learning takes place in many ways. Primary among these 
are two processes we call MEMORY and THINKING . 

d) MEMORY is remembering. That is, you learned it once 
before and now you are asked to conjur it up again. 
Relating or telling experiences falls into this category. 
In general, when the process covers familiar territory, 
it is put into the memory category. 

Examples: -Lecturing, reading, singing, talking 

from experience (show & tell), proving 
they have studied or listened, questions 
such as: What?, When?, Where? 

e) THINKING is taking place when you: 

1) put separate bits of knowledge together and come 
up with something new that you didn't know before, 

2) apply knowledge from »one place to another. 
Example: Solving a problem in math. The student 

knows the mathematical rule hut must 
plug in new numbers and come up with the 
solution. 

f) Thinking usually involves creativity and " imagination. 
In general, when the process covers new material for 
the person doing it, it is called thinking. It is an 
expansion of the existing knowledge base. See 
Transparency CT-2 BREAKDOWN OF MENTAL PROCESS. 

g) This is the breakdown of mental processes for the 
Cognitive Categories: 

MEMORY 

1. Receives information 

2. Repeats it back - checks comprehension 
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THINKING 

3. Puts two received informations together. 

4. Creates something new for the individual. 
Solves a problem correctly. 

h) Research has shown that 85% of all classroom activity 
occurs at the memory level. Of the 15% that occurs at 
the thinking level, the teacher, accounts for 10^. That 
means students generally spend only 555 of their class 
time in thinking activities. 

2) Call for examples that would fit into the general categories 
of memory and thinking. (Exampleo: reporting, discussing, 
solving math problems). List the examples according to 
categories - memory or thinking. 

3) Have trainees make up examples of thinking questions and 
memory questions. Let the group discuss the questions and 
decide in which category (memory or thinking) the questions 
fit. The group must come to a consensus coding for each 
category. Remember, however, that this is a perceptual 
exercise. The group must "hear" each coding alike and be 
able to give reasons to support the category coded. The 
perceptions or "hearings" different from the consensus are 
not wrong, just different. Everyone needs to hear alike in 
order to code reliably. Always refer back to the major 
concepts when in doubt. 

^) Go over the codes together, use Individual copies as well 
as transparency CT-3 COGNITIVE CODES. These points should 
be covered and discussed: 

a) The scale is divided into teacher talk, student talk, 
and affective or emotional behavior. 

b) Under teacher talk the categories are divided into 

. memory and thinking. Categories 1 and 2 are memory; 
categories 3 and 4 are thinking. 

c) Under student talk the categories are divided into 
memory and thinking. Categories 5 and 6 are memory; 
categories 7 and 8 are thinking. 

d) Category 9 is all emotional behavior - those things 
that would be categories 1, 2, 7, and some 6's (such 
as "shut the door") on Flanders' Interaction Analysis. 

e) Category 10 is chaos, silence, or confusion. 
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5) In small groups do Illustration of Categories #1* on the 

Cognitive codes. Bring groups together to discuss codings. 
Each group must be able to explain the reasons behind their 
choices of categories vor each interchange. The whole group 
must come to a consensus coding approximating master codings. 
There are no wrong perceptions, just different . Everyone 
must "hear" alike as coders. Simply explain why you, as 
an instructor, "heard" it the way you did. 

, 6) Have each individual do Illustration of Categories #2.* 

Discuss codings as a group* Each individual must.be able 
to give the reasons for his choices. Same rules apply as 
for procedure 5. (This step may be omitted if trainer feels 
trainees have fully grasped concepts). 

7) Discuss the different patterns which generally occur. These 
points should be covered: 

a) The major patterns are 2-5 (teacher-student on memory 
level), and (teacher-student on thinking level). 

b) Be careful of coding patterns of 2-7, ^-5 which occur 
repeatedly. If this occurs as a repeated pattern, 
there may be something wrong with the codings. The 
students seldom operate on a different intellectual 
plane from the teacher. Re-code the segment (chances 
are the codings are wrong). 

c) As in FlandersS look for the pattern and the most 
significant category occurring in an interval where 
there is more than one type of exchange. 

8) Listen to a segment of the Cognitive training tape.^^ 
Discuss it generally as to major categories appearing in 
the segment. Code a segment on paper at a slow speed 
(5, ^, or 3 seconds) with a metronome or the instructor 
beating time. Repeat coding with various segments until 
trainees achieve the level of proficiency desired.- 

9) If necessary in order to reach reliability standards , the 
following additional training can be conducted: 

a) Work Illustration of Categories #3.^ Repeat procedure 
6. 

b) Repeat procedure 8 with other training tapes.** 



^Appendix A 
**Appendix B 
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OUTCOME INDICES 

For objective 1 ; 

Each individual is physically attentive, alert, and 
participates in the small group work. 

For object; ive 2 and 3 ; 

Each trainee codes the worksheets of a teacher-student 
interchange with 85% accuracy. 

For objective ^ ; 

Each trainee codes on a slow speed such as a 5 second interval.. 
These codings will compare with those of the instructor so 
that there is no more difference than 4 in each category 
total, and for each category that is totaled more than 0 but 
less, than H, each trainee will score at least 1. 

For objective 5 ; 

Each trainee codes any given segment of tape of verbal exchange 
on a three second interval on the Cognitive scale. He does 
this with no more of a difference than h from the trainer's 
codings in the number tallied in each category, except that 
the trainee will identify and record every occurrence of 
categories 3, and 8, 
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AN INSTRUMENT FOR THE ASSESSMENT OP 
COGNITIVE BEHAVIOR IN THE CLASSROOM 



PERSON 



CATEGORY OF BEHAVIOR 



o 



1. Demonstrates knowledge of a fact (Memory 
or, recall and recognition). 

2. Solicits student to demonstrate knowledge 
of a fact. 



1 



c 

•H 
•H 



Uses a fact (thinking) 

Examples: (a) to solve a problem 

to propose an attack on 

a problem. 

(b) to analyze a situation 

Solicits student to use a fact 
(thinking) 

Examples: (a) to solve a problem 

(b) to, analyze a situation 



5. Demonstrates knowledge of a fact 
Cmemory or recall) 

o 6. Solicits someone else to demonstrate knowledge 
§ 'of a fact 



» 



CO 



Uses a fact (thinking) 

Examples: (a) to solve a problem 

(b) to analyze a situation 

Solicits someone else to use a fact 
Examples: (a) to solve a problem 



(b) to analyze a situation 



9. Non-cognitive behavior (Affective behavior) 
10. Silence or confusion 



r 
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CT-I: Cognitive Major Concepts 

(a) Cogn i t i ve means knowledge 

(b) Scale focuses on the intellectual 
acti vi ty--the mental processes . All 
emotional th i ngs--category 7. 

(c) Learning processes categor i zed-- 
Memorv and . Th i nk i ng . 

(d} Memory and remembering. , Relating or 
telling experiences. Covers 
f ami I i or terr i tory . 

(e) Th i nk in g -- i nvo I ve.q creativity, 

imagination; covers new material; 
an expansion of existing knowledge 
base. Thinking: I) combines 
separate bits of knowledge into 
something new, or 2) applies 
knowledge from one place to 
another . 
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CT-2: Breakdown of Mento I Process 



Memory 



Th i nk i ng 



Rece i ves i n format i on 



7. , Repeats it back--checks 
■ comprehension 



3. Puts two received 

i nf ormat i ons together 



V. Creates something new for 
him. Solves a problem 
correctly 




r 
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CT-3: Cognitive Categories 



T 
E 
A 
C 
H 
E 
R 



T 
A 
L 
K 



M 
E 
M 
0 
R 
Y 



T 
H 
I 
N 
K 
I 
N 
G 



1. Demonstrates knowledge of a fact. 

2. Solicits student to demonstrate knowledge of 
a fact. 



3. Uses a fact (analyze or solve). 

4. Solicits student to use a fact (analyze or 
solve) . 



S 
T 
U 
D 
E 
N 
T 



T 
A 
L 
K 



M 
E 
M 
0 
R 
Y 



T 
H 
I 
N 
K 
I 
N 
G 



5. Demonstrates knowledge of a fact. 

6. Solicits someone else to demonstrate knowledge 
of a fact. 



7. Uses a fact (analyze or solve). 

8. Solicits someone else to use a fact, 
(analyze or solve). 



9. Non-cognitive behavior-emotions 
(Affective behavior) 
10. Silence or confusion 



V 
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INTRODUCTION TO PROCESS SCALES 
AND 

MEASURING ENERGY LEVEL 



OBJECTIVES 

1) To maintain a trust relationship so that each trainee 
participates in discussion and contributes his own ideas • 

2) To present the shared basic factors of the affective scales 
so that each trainee can use them as tools to measure the 
affective functioning of teachers (and the resultant effect 
on the students), as well as tools to measure all human 
interaction. 

3) To define energy level according to physical cues so that 
each trainee can discern high and low energy level by 
observing physical signs. 

4) To equate the physical energy cues with vocal energy cues 
so that each trainee can identify energy as being either 
high or low by listening to any given teacher's voice. 

MATERIALS 

1) Training Module 

2) Meaning training tape (Appendix C) 

3) Master ratings of training tapes (Appendix B) 
^) Cassette recorder 

5) Transparency of energy level symbols 

6) Overhead projector 

7) Paper 

8) Pencils 

9) Chalkboard and chalk 



PROCEDURES 

1) The 5 process scales^ for measuring a teacher's affective 
functioning all have several ba'sic factors in common. 
These are: 



*A training module for. each scale (Meaning, Genuineness, 
Success Promotion, Respect, and Student Involvement) 
follows this module. 
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a) All five scales measure the emotional content of the 
classroom. These are intangibles which are reflected 
vocally. 

b) These scales are often called the affective scales. They 
measure the effect that the teacher has on her students. 

c) The effect is always " for better or for worse ." 

d) One of the first steps in using the scales is deciding 
if the teacher is helping or hurting her students 
emotionally and intellectually. 

e) A question you should ask yourself is: "Would I want to 
be in this classroom?" 

f) Each process scale has 5 levels for measuring. Symbols* 
can be used to make first discriminations. 

1) ilndicates that the teacher is interchangeable, 
minimally effective. She is not hurting her students, 
may even be helping them to learn. On a 5 point 
scale, this (±) would at least be a 3.0. 

2) - Indicates that the teacher is hurting her students, 
making them dislike what they are doing. The rating 
for this (-) is 2.0 or lower. 

3) Levels on the scales approximate these. energy levels: 

SEE TRANSPARENCY — Energy/Level Symbols 

— 1.0 Crippling 

- 2.0 Hurting 

± 3.0 Minimally effective 

+4.0 Adding significantly 
++ 5.0 Adding, encouraging, exploring 

In the five process scales, always decide the energy level to 
ciet ermine the range of the scale rating. 

a) Energy level is the key to determining if a teacher is 
helping or hurting. 

Display selected visual aids of persons in 
interaction situations. (These aids could be 
slides, transparencies or even trainer and/or 
trainees modeling high and low energy). 

"Energy level notation adopted from R. R. Carkhuff , The Art of 
Helping . Amherst, MA: Human Resource Development Press, 1^72. 
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b) A working definition to use when measuring energy level 
is: "Energy is the capacity to act." Ask the trainees 
to identify the people ir the situation as either high 
or low energy by Just looking at them. Cues to look for 
are (write them on the board as they are discussed): 
posture, sitting, and standing — are the persons settled 
or ready to move; look at the line of balance in the 
posture — is the body line straight, or slumped and 
leaning; look at the expressions on people's faces — 
either open or closed: look for evidence of confidence, 
defined as having reserves of energy held back and 
waiting to be used, 

c) Equate the physical cues with verbal ones. Call on the 
trainees to translate them. Write the examples on the 
board as they are discussed. Energy may be detected by: 
quickness of response (physical: line of balance), 
appropriate rate of speaking (physical: line of balance), 
balance in volume--nelther too loud nor too soft (physical: 
line of balance), reserves of energy reflected by volume 
and pitch (physical: expressions on faces - aleri;, open, 
confidence). To be a high energy level person, the teacher 
must maintain all of these characteristics at the same 
time. The teacher must maintain her balance. 

d) Listen to taped examples of energy level and Identify 
energy level as high or low* Use the Master training 
tape and the Master ratings. Translate the Hasicer 
ratings into energy symbols (±) . Do not reveal or 
discuss the Master ratings at thi: time. 

e) Further discernments of energy level^ — teacher must have 
all characteristics of high energy level to be rated 

as high. Rate high energy level as ± (at least 3*0) 
and low as - (2.0 or lower). Practice deciding energy 
level and using symbols for all five levels* 

f) Because energy l^vel is a basic ingredient in each 
process scale, ratings between the 4 process scales 
relating to the teacher (Meaning, Genuineness, Success 
Promotion, Respect) probably will not vary more than .7* 

g) Note: In a recent study by Dr. Tom Collingwood of 
Louisiana State University, it was shown that teachers 
who performed the best on physical fitness tests were 
rated higher according to these scales. Energy level 
does have a direct bearing on a teacher's functioning 
in the classroom. 
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h) Repeat steps d & e until 90% accuracy Is obtained* 



OUTCOME INDICES 

For objective 1 ; 

Each trainee maintains an alort posture and participates in 
the discussions. 

For objective 2 ; 

Each trainee listens to an audio tape and rates the affective 
functioning of the teacher as either helping or hurti-v^ the 
students. 

For objective 3 and 4 ; 

" Each trainee identifies energy level as being either high 

(±) or low (~) for any given segment of a teacher's vocal 
behavior with 90% accuracy. 
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ENERGY/LEVEL SYMBOLS 

1.0 Cr i pp I i ng 
- 2.0" Hurt i ng 
+3.0 Minimally Effective 
+ V.O Adding Significantly 

s ^ 

■++5.0 Adding, Encouraging, 
Exploring 

/ 

/ 



A SCALE FOR THE iVIEASUREMENT OP A TEACHER'S 
UNDERSTANDING OF THE MEANING OF CLASSROOM EXPERIENCES 

FOR HER STUDENTS 



OBJECTIVES 

To maintain a trust relationship so that each trainee 
participates in discussion and contributes his own ideas. 

To present the Meaning scale so that each trainee is able 
to use it as a tool to measure a teacher's empathy with her 
students as well as a tool to measure empathy in all human 
interactions . 

To present the 5-level measure of the Meaning scale 
incorporating energy level so that each trainee can 
distinguish between broad levels with 85% accuracy. 

To refine each trainee's discernment so that he can identify 
not only 5 levels but three sub-levels (.3, .5, .7) within 
each level to ± .3 agreement with the master ratings. 



MATERIALS 

Understanding of Meaning scale (copies for each participant) 

Meaning training tape (Appendix C) 

Master ratings of training tape (Appendix B) 

Cassette recorder 

Chalkboard and chalk 

Pencils 

Paper 

Transparency 
Overhead projector 



PROCEDURES 

Review energy level identification by playing a sample tape 
segment and deciding if energy is high (±) or low (-) . 

Move into a didactic presentation of A SCALE FOR THE MEASURE- 
MENT OF A TEACHER'S UNDERSTANDING OF T HE MEAtjINS 6F CLAsSROT 
EXPERIENCES FOR HER STUDENTS' . Read the scale together, a 
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trainee taking each level. Utilize individual copies of 
the scale. Discussion should follow. These are the 
important points to be covered: 

a) The title indicates the basic purpose of the scale. 

b) Energy level is applicable to the Meaning scale in 
these ways: 

1. The teacher uses a meaningful variation of tone. 

2. The teacher has reserves of energy — energy that 
she calls on as indicated by: 

a) well modulated volume; not screaming, 
shouting, whispering, etc. 

b) well modulated pitch. (Be aware that pitch 
is sometimes the result of the teacher's 
physical being. Pitch should be marked 
down only if it is in the extreme — squeaky, 
raspy, whining, etc.) 

3. The teacher's rate of speaking is appropriate to 
the lesson material — not monotonous, not falling 
into a sing-songy pattern. 

4. The teacher's quickness of responses-the 
spontaneity she exhibits - is appropriate. 

c) Energy level is a good clue to how the teacher will 
be rated. To be a level 3, the teacher must have 
high energy—must exhibit all of the characteristics 
listed above. 

d) Praise is significant on this scale. (Flanders' 
category 2; 9 on Cognitive). Some use of praise is 
necessary for the teacher to be a level 3 > fraise does 
not always have to be "Good". It may be other feeling 
words or tone of voice. The best praise is more than 
Just a "Good" response. It is specific and appropriate 
to the situation. A level 3 on this scale responds 

at least with a "Good". 

e) For a teacher to be above a level 3 on this scale, she 
must approach interchangeable respcfnses with her 
student, using some feeling v/ords, and much patience. 
To be a level ^ or above > she must make interchangeable 
responses . 
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f) Use the transparency. Meaning Contlnua, to put scale into 
a capsule for trainees to remember, 

3) Listen to two extremes of Meaning on the training tape. Have 
the trainees rate energy level. Translate the energy symbols 
into numbers: — 1, - 2, 1 3, + 1|, ++ 5. Explain that this 

is the general level rating on the scale. Take into consideration 
the teacher qualities previously listed: praise, patience, 
feeling words, and interchangeable responses. Have trainees read 
the scale and adjust their level ratings to include these 
qualities. The level ratings should correspond to the 
description of the teacher's verbal behavior. Reveal only 
the broad levels of the master ratings (1.0, 2.0, 3.0, 4.0. 5.'0) 
to the trainees. 

4) Listen to various segments on the training tape and identify 
levels. Discuss when necessary. ' There is no "wrong" answer. 
When disagreements occur, call on various trainees to explain 
why they heard it the way they did. Explain the master 
ratings and that the' purpose is for all to be able to hear it 
this way. Refer discrepancies to Meaning Continua. 

*5) Explain sub-levels. Teachers may not fit levels exactly 

according to the descriptions in the scale — they probably 

won't. Therefore, there are sub-level ratings. If the 

teacher is a little better than the description of the level, 

she is that level plus .3. If she is about half-way in between 

two levels, she is the lower level plus .5. If she is not 

quite the upper level, she is the lower level plus .7. Listen 

to taped examples and discern: (1) energy level, (2) broad 

level on Meaning scale, (3) scale sub-level. Discuss inter- 
]/6x&uionsnips • 

6) If necessary in order to reach reliability stand ards^ repeat 
procedure 5 with other training tapes, 



OUTCOME INDICES 

For objective 1 ; 

Each trainee maintains an alert posture and participates In 
the training session. 

For objective 2 and 3 : 

Each trainee distinguishes between broad levels to 85% 
accuracy with master ratings. 



*Step 5 should not be undertaken until trainees can agree 
on broad level ratings. 
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RELIABILITY INDICE 



For objective ^t ; , 

Each trainee Identifies the levels and sub-levels to - .3 
agreement with master ratings for any given tape segment. 
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A SCALE FOR THE MEASUREMENT OP A TEACHER'S UNDERSTA JDING 
OP THE MEANING OP CLASSROOM EXPERIENCES POR HER STUDENTS 

Neither the tone quality nor the words of the teacher's 
verbal conununication convey any feelings, and/or she 
responds .inaccurately to the meaning of the students' 
experiences. 
Examples: 

(1) The tone of the teacher's voice is flat or 
monotonous . 

(2) The teacher says, "You enjoyed that" after a 
student's performance indicates obvious dislike 
for the activity. 

The tone quality of the teacher's verbal communication 
conveys slight evidence of feelings which are only some- 
what appropriate to her student's experience. She uses 
no words to explicate her feelings. 
Examples : 

(1) The teacher's voice is very subdued and controlled. 

(2) The teacher says, "Let's hold it down" after a ■ 
student expresses Joy with the activity. 

The -tone quality of the teacher's verbal communication 
conveys feelings which are quite appropriate to her 
students' experiences. She is "with" her students. 
However, she uses no words to explicate her feelings. 
Examples : 

(1) The teacher's voice matches that of her students. 
She neither adds nor detracts from the meaning 
of their experience. 

(2) The teacher says, "Good" after a student demonstrates 
appropriate Joy with the activity. 

The tone quality of the teacher's verbal communication 
conveys feelings which are appropriate to her students' 
experiences. Additionally, she uses mild words to describe 
the feelings. 
Examples: 

(1) The teacher adds slightly to the meaning of the 
students' experiences by appropriate words. 

(2) The teacher says, "Good, you seemed to really 
enjoy that!" After a student demonstrates 
appropriate Joy with the activity. 

The tone quality of the teacher's verbal communication 
conveys feelings which are appropriate to her students' 
experiences. Additionally, she uses "strong" words to 
describe her feelings. 
Examples : 

(1) The teacher adds a great deal to the meaning of 
the students' experiences by aopropriate words. 

(2) The teacher says, "Great I felt like you were 
going to dance you lined that so much!" after 
a student demonstrates appropriate Joy with the 
activity. • 4t) 
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A SCALE FOR THE MEASUREMENT OP A TEACHER'S • 
GENUINENESS IN HER CLASSROOM INTERACTION WITH STUDENTS 



OBJECTIVES 

1) To maintain a trust relationship so that each trainee 
participates in the discussion and contributes his own ideas. 

2) To present the Genuineness scale so that each trainee is able 
to use it as a tool to measure a teacher's congruence with 
her students, as well as a tool to 'measure congruency in all 
human interactions. 

3) To present the 5-level measure of Genuineness incorporating 
energy level so that each trainee .can distinguish between 
broad levels with Q5% accuracy. 

4) To refine trainee's discernment so that each can identify 
not only 5 levels but three sub-levels (.3, .5, .7) within 
each level to ± .3 agreement with the master ratings. 



- MATERIALS 

1) Genuineness scale (copies for each trainee) 

2) Meaning training tape - (Appendix C) 

3) Master ratings for training tape - (Appendix B) 

4) Cassette recorder 

5) Chalkboard and chalk 

6) . Pencils 

7 ) Paper 

8) , Transparency 

9) Overhead projector 



PROCEDURES 

1) Review energy level identification by playing a sample tape 
segment and deciding if teacher is high or low on energy. 

2) Move into didactic presentation of A SCALE FOR THE MEASURE- 
MENT OF A TEACHER'S GENUINENESS IN KER CLASgROOM INTERA cI-ION 

WITH STUDENTS^ Read the scale together, a trainee taking 

each level. [Ttilize individual copies of the scale. Follow 
with discussion and cover these points: 
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a) The teacher's energy level is very important to this 
scale in these ways: 

1) There is a meaningful variation of tone. 

2) There is a quickness of responses. This is 
spontaneity — the teacher addresses herself 
openly and fully to the behaviors in herself 
and her students as they arise. 

b) Ask yourself — "Does the teacher sound as if she is 
portraying a role as a school teacher rather than 
being a real person on a one-to-one basis with her 
studsnts?*' If she does this, she is below a level 3. 

c) Use the transparency of the Genuineness Continua 
and discuss the use of pronouns (as shown on the 
transparency) as a cue to genuineness and level 
ratings for role playing. Discuss the other factors 
shown on transparency and their inter-relationships. 

3) Listen to two extremes of Genuineness from the training 
tape. Have the trainees rate energy level. Then, translate 
energy symbols into numbers: — 1, - 2,i35 '^'^ 5. 
Explain that this is the general level rating on the scale. 
Take into consideration the qualities listed under procedure 
2: spontaneity, openness, and role playing. Read the scale 
and have trainees make adjustments to include those qualities 
according to the description of the teacher under each level. 
Reveal and discuss the master ratings. 

4) Listen to various segments on the training tape, identifying 
the levels. Discuss when necessary. There is no wrong 
answer. This is a perception exercise and each perceives 
things differently. Call on various trainees to explain 
why they heard it the way they did. Explain the master 
ratings and that the purpose is for all to be able to hear 

it, to perceive it, alike. Refer discrepancies to Genuineness 
Continua. 

*5) Explal?r->the scale sub-levels. Teachers may not fit levels 
exactly according to the descriptions in the scale. They 
probably won't. Therefore, there are sub-level ratings. 
If a teacher is a little better than the description of the 
level, she is that level plus .3. If she is about half way 
in between two levels, she is the lower level plus .5. If 



*Step 5 should not be undertaken until trainees can agree 
on broad level ratings. 
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she is not quite the upper level, she is the lower level plus 
.7. Listen to tapes for examples and identify: (1) energy 
level, (2) level on scale, (3) scale sub-level* 

6) If necessary for reliability standards ^ repeat procedure 5 
with other training tapes. 



OUTCOME INDICES 

For objective 1 ; 

Each trainee maintains an alert posture and participates 
in the training session. 

For objective 2 & 3 ; * 

Each trainee distinguishes between broad levels to 85% 
accuracy with master ratings. 



RELIABILITY INDICE 

For objective 4 ; 

Each trainee identifies levels and sub-levels to i .3 
agreement with master ratings for any given tape segment. 
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A SCALE FOR THE MEASUREMENT OP A TEACHER »S GENUINENESS 
IN HER CLASSROOM INTERACTION WITH STUDENTS 



Level 1. All of the teacher *s verbal communications are ritual- 
istic. They seem to be mechanical or practiced* 
Examples : 

(1) The teacher sounds like a "school reacher." 
("teacherish voice") 

(2) The teacher slowly and/or mechanically says, 
"Turn to page 99 and begin reading silently." 

Level 2. Most of the teacher *s verbal communications are ritual- 
1st ic, but a few are somewhat spontaneous. 
Examples : 

(1) The teacher sounds like a "school teacher" most 
of the time, but occasionally she sounds like 
she is having a "normal" conversation. 

(2) The teacher rather slowly says, "Turn to page 
99 and begin reading silently," but she gives 
evidence of some (though not much) vitality. 

Level 3. The teacher* s verbal communications are about equally 
distributed between ritualistic and spontaneous. 
Examples : 

(1) The teacher sounds like a "school teacher," 
about half the time, while for the other half 
she seems to be having a "normal conversation." 

(2) The teacher cays, "Let's turn to page 99 and 
begin reading," and she gives evidence of normal 
vitality. She is neither wildly enthusiastic 
nor dull. 

Level 4. Most of the teacher's verbal communications are 
spontaneous, but a few are ritualistic. 
Examples : 

(1) The teacher only rarely sounds like a "school 
teacher." Most of the time she sounds as 
though she is engaging in "normal" conversation. 

(2) The teacher says, "Let's turn to page 99> and 
would anyone like to read to us?" 

Level 5. All of the teacher's verbal communications are 

spontaneous. They are neither mechanical nor practiced. 
Examples: 

(1) The teacher always sounds like she is having 
"normal" conversatione 

(2) The teacher says, "What do you want to read 
today? Does anyone know an exciting story?" 
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GENUINENESS CONTINUA 



Role 



1-0 2.0 3.0 4.0 5.0 

! « _i !_ 

"Ole Role I^le Person Person 

to to to to to 

"Object" Role Person Person Person 

Frequently Always 



1.0 



Third 
Person 



Use of Personal Pronouns 
2.0 3.0 4.0 



Meaningless 

«We" + 
Third Person 



TouV 



5.0 
J. 



T7i You + We 

(meaningless) (meaningful) 

I (meaning- I (meaning- 
less) fui) 



Tou 
+ 
I 



1.0 

Ritualistic 
lOOIt 



2.0 

L__ 



Communication 
3.0 

1 

50-50 



4.0 
1 



5.0 

spontaneous 
lOOJt 



Energy Level 

1.0 2.0 3.0 4.0 5.0 

^ • i — : 1 L 

+ + ++ 



ERIC 52 



0 



A SCALE FOR THE MEASUREMENT OP THE PROCESS 
OP SUCCESS PROMOTION 



OBJECTIVES 

1) To maintain a trust relationship so that each trainee 
participates in the discussions and contributes his ovm ideas. 

2) To present the Success Promotion scale so that each trainee 

is able to use it as a means to measure a teacher's priorities 
in her classroom experience* 

3) To present the 5 level measure of Success Promotion 
incorporating energy level so that each trainee can 
discriminate between broad levels with B5% accuracy^ 

4) To refine trainee's discernment so that each trainee can 
identify not only 5 levels but three sub-levels (.3, .5, .7) 
within each level to ± ^3 agreement with master ratings. 



MATERIALS 

1) Success Promotion scale (copies for each participant) 

2) Meaning training tape (Appendix C) 

3) Master ratings of training tapes (Appendix B) 

4) Cassette recorder 

5) Chalkboard and chalk 

6) Pencils 

7) Paper 

8) Transparency 

9) Overhead projector * 



PROCEDURES 

1) Use individual copies of A SCALE FOR THE MEAS UREMENT OF THE 
PROCESS OF SUCCESS PROMOTION and read it ^Together Ask 
trainees to explain each level. Lead a discussion to cover 
these points: 

a) The teacher's energy level applies to the Success 
Promotion scale in these ways: 
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1. Quickness of the teacher's responses — The teacher 
must have energy to be alert to cues. 

2. Capacity to move— flexibility. The teacher must be 
able to move freely from onfe subject matter to 
another. If she cannot relate to her students 
except on a specified subject matter, she has low 
energy. Many teachers lecture all of the time, 
sticking to their lesson plans because they don't 
have enough energy to relate to students outside 

of the subject matter. 

3. Reserves of energy — The teacher must be confident of 
her energy which she can call on to meet her students 
where they want to go. 

b) This scale indirectly measures a teacher's anxiety. At 
level 1 and 2 the teacher is most anxious about covering 
her lesson material. Her anxiety decreases at level 3 and 
disappears through level 5. 

c) At level 1 and 2, the teacher is more concerned with her 
lesson plan than with what her students are learning. She 
ignores questions and may use many criticisms (Flanders' 
category 7), 

d) "Off the subject" does not indicate chaos, but the subject 
that the class strays to is important and relevant to 
students and/or their learning goals. 

e) Ask yourself — "How aware is the teachor of what her 
students want to learn? Does she have a place for those 
desires in her classroom?" The teacher should be 
interissted in each individual, in what he wants to learn, 
and in what he is. learning. 

2) Listen to two extremes of Success Promotion on the training 
tape. Have trainees rate energy level. Translate energy 
symbols into numbers: — l, - 2, ± 3, + i|, ++5. The 
trainees should read the scale taking into consideration the 
points listed under procedure 1: flexibility, teacher's 
goals versus student goals, and awareness of student needs. 
After reading the scale, the trainees should make adjustments 
in rating levels to include these qualities. Reveal and 
discuss master level ratings. 

3) Use the transparency — Success Promotion Continua to 
summarize. Discuss inter-relationships. 
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4) Listen to various segments on the training tape and identify 
levels. Discuss when necessary* There is no wrong ans wer. 



Call on various trainees to explain why they heard it as 
they did. Explain the master ratings and that the purpose 
is for all to be able to hear it alike so as to match the 
master ratings. Refer dis^prepancies to Success Promotion 
Continua . 

^5) Explain the scale sub-levels. Teachers may not fit levels 
exactly according to the descriptions in the scale. They 
probably won't. Therefore^ there are sub-level ratings. , 
If the teacher is a little better than the description of 
the. level, she is that level plus .3. If she is about 
half way between two levels, ,?he is the lower level plus 
- .5. If she is not quite the upper level, she is the 
lower level plus .7» Listen to taped examples, and 
identify: (1) energy level, (2) scale level, (3) scale 
sub-ievel. 

6) If necessary for reliability standards , repeat procedure 
5 with other training tapes. 



OUTCOME INDICES 

For objective 1 : 

Eac-h trainee maintains an alert posture and participates 
in the training session. 

For ob.iective 2 and ^ ; 

Each trainee distinguishes between broad levels to Q5% 
accuracy with master ratings. 



RELIABILITY INDICE 

For objective 4 ; 

Each trainee identifies scale level and sub-level 
ratings to i .3 agreement with master ratings for any 
given tape segment. 



" *St6p 5 should not be undertaken until trainees can agree 
on broad level ratings . 
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A SCALE FOR THE MEASUREMENT OP THE PROCESS OP SUCCESS PROMOTION 



Level 1. The teacher's verbal behavior is directed exclusively 
toward accomplishing her goals without regard to those 
of her students. 
Examples: 

(1) The teacher ignores students' questions. 

(2) The teacher punishes student behavior which she 
deems is away from the lesson. She seems to 
pursue her pre-established schedule rigidly. 

Level 2. The teacher's behavior is directed primarily toward* 
accomplishing her goals, but occasionally she acts to 
help students achieve their self-directed goals. 
Examples: 

(1) The teacher responds to a few student questions, 
but ignores most of them. 

(2) The teacher occasionally allows a student to 
discuss something "off the subject.". She seems 
very aware of being-in-charge of the group and 
of covering a prescribed amount of the material. 

Level 3- The teacher's verbal behavior is directed toward 

accomplishing her goals about 50% of the time and the 
students' self -directed goals about 50% of the time. 
Examples: 

(1) The teacher responds to about half of the 
students ' questions . 

(2) The teacher gets "off the subject" about 5058 of 
' the time in the sense that she enters into a 

dialogue with students. She seems to feel in 
charge of a group and concerned about covering a 
prescribed amount of material. However, she does 
not seem anxious about it. 

Level 4. The teacher's verbal behavior is directed primarily 

toward helping her students accomplish their self-directed 

goals without regard to her own goals. 

Examples: 

(1) The teacher responds to most of the students' 
questions. 

(2) The teacher "gets off the subject" easily. In 
fact, she seems to enjoy doing so and sustains 

... .„ it by eliciting a large number of student 

initiated statements. She gives only slight 
evidence of either being in charge or being 
limited by the amount of material to be covered. 
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Level 5. All of the teacher's verbal behavior is directed 

toward helping the students accomplish their self- 
selected goals without regard to her own goals. 
Examples : 

(1) The teacher's approach is geared to cope 

with all the students' questions as they state 
them. 

(2) The teacher's subject matter consists solely^ 
of the process of helping students accomplish 
their goals. She dialogues with them freely . 
and openly without any evidence of being 
limited either by concerns about being in-charge 
or by the amount of material to be covered • 
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SUCCESS PROMOTION CONTINUA 
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A SCALE FOR THE MEASUREMENT OP RESPECT 
PROVIDED BY THE TEACHER IN CLASSROOM INTERACTION 



OBJECTIVES 

1) To maintain a trust relationship so that each trainee 
participates in discussions and contributes his own ideas. 

2) To present the Respect scale so that each trainee is able 
to use it as a tool to measure a teacher »s respect for her 
students' learning abilities as well as to measure the 
amount of respect in all human interactions. 

3) To present the 5-level measure of Respect incorporating 
energy level so that each trainee can distinguish between 
broad levels with 85% accuracy. 

4) To refine the trainee *s discernment so that he can identify 
not only 5 levels but three sub-levels (.3, .5, .7) within 
each level to ± .3 agreement with master ratings. 



MATERIALS 

1) Respect scale (copies for each participant) 

2) Meaning training tape (Appendix C) 

3) Master ratings of training tape (Appendix B) 

4) Cassette recorder 

5) Chalkboard and chalk 

6) Pencils 

7) Paper 

8) Transparency 

9) Overhead projector 



y PROCEDURES 

1) Utilize individual copies to present A SCALE FOR THE MEASURE^ 
MENT OF THE RESPECT PROVIDED BY THE TEACHER IN CLASSROOM 
INTERACTION ^ Read the scale together. Have a trainee explain 
each level. Lead a discussion to cover these points: 
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a) The teacher's energy level applies to this scale in these 
ways: Reserves of energy— this aspect of energy applies 
in two extremes. The teacher who i. not able to retain 
or conserve her energy, runs over her students in her 
hastiness to put our information. She ignores students 
and comments because she is so involved in what she has 
to say. Also, there is the teacher who has no reserves 
of energy to call upon. She fills up class time with her 
low level usage of time so that she won't have to meet 
students' problems or ideas. Both of the extremes in 
reserves of energy are classified as low energy level 
and are never rated above the 2 level on this scale, 

b) When rating this scale, ask yourself: "Does the teacher 
expect the student to learn, to answer her questions? 

Is she interested in the individual student, the slow as 
well as the quick?" 

c) The Cognitive scale goes hand-in-hand with this one. 
Respect levels 1-3 operate only on Cognitive categories 
1, 2, 5, and 6. Respect level 3 may be accompanied by 
a few 4-7 patterns from Cognitive, but not many. Level 
4 and 5 teachers on the Respect scale also use many 
instances of Cognitive categories 3, 4, 7, and 8. These 
upper levels may also be accompanied by categories 1, 2, 
and 3 on Flanders ' . 

2) Listen to two extremes of Respect. Have the trainees rate 
energy level and translate the energy symbols into level 
numbers (— i, - 2, i 3, + 4, ++ 5) . Trainees should then 
read the Respect scale and cake into consideration the points 
listed under procedure 1. They should make adjustments in 
level ratings accordingly. Ask the trainees to share aloud 
the level of their ratings for Respect. Reveal and discuss 
master level ratings. 

3) Use the transparency — Respect Continua to show the inter- 
relationships of various aspects of the Respect process, 

4) Listen to several segments on the training tape and identify 
levels. Discuss when necessary. There is no "wrong" answer. 
Ask the trainees to explain why they heard it the way they 
did. Explain the master ratings and that the purpose is for 
all to hear it alike in the same way. Refer discrepancies to 
chart of the Respect Continua. 
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5) Explain scale sub-levels. Teachers may not fit levels exactly 
according to the descriptions in the scale. They probably 
won't. Therefore, there are sub-level ratings. If the 
teacher is a little better than the description of the level, 
she is that level plus .3. If she is about half way in between 
two levels, she is the lower level plus .5. If she is not 
quite the upper level, she is the lower plus .7. Listen to 
taped examples, identify: (1) energy level, (2) Respect level, 
and. (3) Respect sub-level. 

6) If necessary for reliability standards , repeat procedure 5 
with other training tapes. 



OUTCOME INDICES 

For o.bjective 1 ; 

Each trainee maintains an alert posture and participates in 
the training session. 

For objective 2 and 3 : 

Each trainee aistinguishes between broad levels to Q5% accuracy 
with master ratings. 



RELIABILITY INDICE 

For objective 4 ; 

Each trainee identifies levels and sub-levels to i .3 
agreement with master ratings for any given tape segment. 



*Step 5 should not be taken until trainees can agree on broad 
level ratings. 
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A SCALE FOR THE MEASUREMENT OP THE RESPECT PROVIDED 
BY THE TEACHER IN CLASSROOM INTERACTION 

Level 1. The teacher communicates a clearly negative regard for 
the students' individual abilities to learn. 
Exrn'ples: 

(x) '"he teacher structures the situation so the 
student takes little or no active part in the 
learning process; i.e., lectures or gives 
unnecessarily detailed, repetitive directions, etc. 

(2) The teacher seems to mean it when she says, "I 
don't expect you to learn this.- It's too 
difficult for you." 

Level 2. The teacher communicates a somewhat negative regard for 

the students' individual abilities to operate effectively 
in learning situations involving memory and recognition 
(level 1 of Bloom's Taxonomy, 1967). 
Examples : 

(1) The teacher structures the learning situation so 
that the student can appropriately respond only 
by rote, but often fails to allow enough time 

for even that response; i.e., answers own questions 
or calls for "help" with the answer. The teacher 
communicates doubt that the students will be able 
to participate "correctly." 

(2) The teacher says, "Even this is too difficult 
for many of you." 

Level 3. The teacher consistently communicates a positive regard 
for the students' individual abilities to operate 
effectively in learning situations involving memory and 
recognition (level 1 of Bloom's Taxonomy), but not with 
the higher intellective processes; i.e., creativity, 
problem-solving, judgment. 
Example^s : 

(1) The teacher structures the situation in such a 
manner that the students are expected and 
encouraged to respond at level 1 of the cognitive 
processes, but responses at higher levels are not 
appropriate. 

(2) The teacher says, "I'll do the thinking. You pay 
attention and learn." 
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Level k. The teacher consistently communicates a positive regard 
for the students* abilities to operate effectively in 
learning situations involving memory and recognition 
(level 1 of Bloom* s Taxonomy), and occasionally allows 
the students to explore the higher intellective processes, 
Examples: 

(1) The teacher sometimes structures the situation 
so that she expects responses at higher levels. 
They are considered appropriate and are received 
by the teacher as worthwhile contributions to 
the learning process* 

(2) The teacher says, "Let's not strain our brains, 
but take time to think of some new ways to do 
that . " 

Level 5. The teacher consistently communicates a positive regard 
for the students* abilities to operate effectively at 
all intellective levels. 
Examples: 

(1) The teacher structures the learning situations 
so that she expects responses at higher Icvel^^j. 
They -are always appropriate and encouraged. 
Such responses are re'ceived by the teacher as 
worthwhile contributions to the learning process. 

(2) The teacher says, "1*11 bet we can think of a 
hundred new ways to do that." 
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I 



None 



2.0 



Little 



Respect 
3.0 



Some 



^1.0 



Good 



5.0 
I 

"RTgh 



1.0 



2.0 



"Able" Projection 

3.0 /|.o 
i I 



5.0 

I 



can't 

do 
ever 



can't 
do 
often 



can 
do 

sometimes 



can 
do 

frequently 



can 
do 
always 



Expectancy 

1-0 2,0 3,0 H.O 5. Of 

thinking 



Nothing Tacts ' ftcti itcts ~^ 



maybe + 

thinking 



Appropriate Energy Level 
1-0 2.0 3.0 H.o 5.0 
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A SCALE FOR THE MEASUREMENT OP STUDENT INVOLVEMENT 



OBJECTIVES 

To maintain a trust relationship so that each trainee 
participates in the discussion and contributes his own ideas • 

To present the Student Involvement scale so that each trainee 
is able to use it as a tool to measure the students' interest 
and participation in the classroom activities. 

To present the 5-level measure of Student Involvement 
incorporating energy level so that each trainee can distinguish 
between broad levels with Q5% accuracy. 

To refine the trainee's discernment so that he can identify 
not only the 5 scale levels but three sub-levels (.3, .5, •?) 
within each level to i .3 agreement with master ratings • 



MATERIALS 

Student- Involvement scale (copies for each trainee) 

Student Involvement training tape (Appendix C) 

Mae^ter ratings for training tape (Appendix B) 

Cassette recorder 

Chalkboard and chalk 

Pencils 

Paper 

Transparency 
Overhead projector 



PROCiZDURES 

Use individual copies to introduce A SCALE FOR THE MEASURE^ 
MENT OF STUDENT INVOLVEMENT > Lead a discussion to cover these 
points: 

a) This scale is different from the other four process scales • 
This is the only one which measures the students • This 
scale is not directly concerned with the teacher • 



65 



60 



b) Energy level is applied to students' behavior as 

demonstrated by their verbal behavior. Look for the 
same signs of energy as looked for in teachers. (Review 
Energy Level if necessary). 

2) Listen to Student Involvement training tape segments. Rate 
the energy level of students. Translate energy symbols into 
numbers (— 1, - 2, ± 3, + 4, ++ 5). if you can't hear the 
students in the whole segment, ask yourself how you would feel 
sitting there listening to that teacher. What kind of energy 
would you have? Then reread the scale and take into consideration 
the discussion and points listed under procedure 1, before 
assigning a rating level. Ask trainees what level of ratings 
they gave. Discuss why they heard it the way they did. Reveal 
and discuss master level ratings. 

3) Show transparency— Student Involvement Continua. Discuss. 

4) Listen to various segments on the training tape, identify the 
levels. Discuss when it seems necessary. There is no wrong 
answer. Call on various trainees to explain why they heard it 
as they did. Explain the master level ratings, and that the 
purpose is for all to hear it alike according to the master 
ratings. Refer discrepancies to Student Involvement Continua. 

*5) Explain scale sub-level ratings. A teacher may not fit levels 
exactly according to the descriptions in the scale. Most of 
them probably won't. Therefore, there are sub-level ratings. 
If the students are a little better than the description of 
the level, they are that level plus .3. If they are about . 
half way in between two levels, they are the lower level plus 
.5. If they are not quite the upper level, they are the lower 
level plus .7. Listen to taped examples and identify: (1) 
energy level, (2) scale level, and (3) scale sub-level. 

6) If necessary for reliability standards , repeat procedure 5 
with other training tapes. " 



OUTCOME INDICES 

For objective 1 ; 

fiacn trainee maintains an alert posture and participates in 
the training session. 



*2tep 5 should not be taken until trainees can agree on broad 
levels of ratings. 
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For objective 2 and 3 : 

Each trainee distinguishes between broad scale levels to 
85^ accuracy with master ratings. 

RELIABILITY INDICE 

■ ^ 

For objective 

Each trainee Identifies the scale levels and sub-levels to 

± .3 agreement with master ratings for any given tape segment. 
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A SCALE FOR THE MEASUREMENT OP STUDENT INVOLVEMENT 



Level 1, 



Level 2, 



Level 3, 



Level 4. 



Level 5. 



The student (s) is not involved in the classroom activity 

prescribed by the teacher. 

Examples: 

(1) He expresses a strong dissatisfaction with the 
present activity. 

(2) He makes a remark unrelated to present activity. 

The student (s) participates about half of the time in 
the activity prescribed by the teacher. 
Examples : 

(1) He makes a response to the activity and follows 
it by one unrelated to it. 

(2) He expresses mild dissatisfaction with the 
present activity. 

The student (s) participates in the class actively, but 
only within the prescribed rules. 
Examples : 

(1) All his responses are relaOed to the class activity, 
but he seems merely to be going along with the game. 

(2) He expresses neither satisfaction nor dissatisfaction 
with the activity. 

The student (s) participates enthusiastically in the class 
activity, but sticks pretty much to the rules established 
by the teacher. 
Examples : 

(1) All his responses are related to the class activity, 
and he seems to enjoy it. 

(2) He expresses mild satisfaction with the activity. 

The student (s) participates enthusiastically in the class 
activity and goes beyond the rules established by the 
teacher. 
Examples : 

(1) All his responses are related to the class activity, 
and his enthusiasm is reflected in his exploration 
of new ideas stemming from it. 

(2) He expresses strong satisfaction with the activity. 
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STUDENT INVOLVEMENT CONTINUA 
Student Involvement 



1.0 
I 



2.0 
I 



3.0 
I 



l\.0 
I 



5.0 
I 



Chaos 
(active 
resistance) 



Passive 
resistance 



Control 



Participation Involvement 



1.0 
I 



2.0 
I 



Student Directionality 

3.0 i|.0 



5.0 
I 



Away from 
teacher! s or ^ 
learning goals ^ 



Please 
teacher 



Learning 
goals 



Student Contribution Toward Learning Goals 



1.0 


2.0 
1 


3.0 
1 


^.0 
1 


5.0 
1 


Detractor 


Observer 


Participant 
Learning Energy 


Contributor 


Leader 


1.0 
1 


2.;0 

1 


3.0 
1 


^.0 
1 


5.0 
1 


None 


Little 


Some 
+ 

Exhibited Energy 


More 
+ 


A lot 
++ 


1.0 
1 


2.0 
1 


3.0 
1 


^.0 
1 


5.0 
1 


+ 

(flight) 




+ 


+ 


++ 

Action 
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APPENDIX A 



i 

Segment 


Illustration of Categories #1 
High. School English Class ^ 


u t 
<U 0 
•0 & 

q 0) 

H Rt 


Cogniti\^e 
Catecjorv! 


A 


Teacher: I think you were very perceptive in writina 
your essays. . . 


2 


1 


B 

- 


Teacher: and I think that from the essays you have 
written we can develop in the classroom a fine 
scale that would really be meaningful^ one that 
^ would mean something to students who were .trying 
to evaluate their teacher's performance and mean 
something to the teacher who is using the 
evaluation system. 


• 3- 
5» 


1 


C 


Teacher: Now* the first thinK we do* some of you 
said your best-liked teacher and your best 
teacher were not the same person. Would you 
clarify that? Someone? 


4 


A 


D 


Student: Sometimes we have a real ccood time in 
A: class but we don't learji anything. 


8- 
9 


7 


E 


Student: Sometimes we have a bad time but we learn 
B: a great deal. 


9 


7 


P 


Teacher: Is it possible to have a Kood time and 
still learn somethings though? 


4 


4 


6 


ouuaeni/ : we snouia. 
C: ^ 


8 


7 


H 


Teacher: Does that happen often? 


4 


2 


I 


student : We feel like we are wasting our time if 
D: we don't learn anything. 


9 


7 


J 


Teacher: I think that is a very important state-- 
ment. It's nice to have fun but you are 


2- 

-3— 




here to learn. Kightv 







•Moves from category 3 to category 5. 
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4J 

a> 


(Cant'd) 

— - - Illustration of Categories #1 ^ - 

High School English Class 


F lander ' s 
Cateqory" 


Cognitive 
Category 


K 


Student: If I want to have fun, I tto out and drink 
oi Deer. 


9 


5 


• L 


Class: (LOUD. LONG LAUGH) 


10 


r 

10 


M 


Student: Go to" the swimminK hole or some nlace. 
C: 


9 


5 

it 


N 


Teacher: Shouldn't learnine be fun? 


4 


4 


0 


Student: It should be fun but not funnv. 


8 


7 „ 


P 


Teacher: I think you have a valid noint in that it 
shouldn't be a party; that learning should 
take place. 


3 


1- 
3 


Q 

- 


Student: If we ko home and it has been fun in class^ 
A: but then we can't work the problems and we 
don't know what to do and make bad grades, 
then it's not so funny. 


9 


- i 

7 


R 


.Student: Sometimes I worry about next vear. I'm 
C: having fun this year, but I bet next year it 
is going to be hard. 


9 


1 


S 


Teacher: You are afraid you won't be orepared for 


1 


9 


T 


Student: Yes» and I want my teacher now to teach 
- the things I need to know then. 


9 


7 


U 


Teacher: O.K., you all indicated on your papers that 
you think that the teacher should be evaluated 
on the knowledge of the subject matter even 
though most of you think to yourself that most 
of your teachers were well prepared in this area. 


3- 

5 


1 
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Segment 


(Cont«d) 
Illustration of Categories #1 
High School English Class 


Plander's 
Cateaorv 


Cognitive 
Category 


V 


Teacher: Are there any disacreements with that state- 
ment? 


•* 


A 


W 


Student: Some teachers may think they are well 

C; prepared but they aren't planning on our reading 
the book because the book may say one thing and 
then they may say something else completely 
different and then if you point it out to them 
they get angry. 


8- 
9 


7 


X 


Teacher: That is a good observation and it ... ' 


2 


9 


Y 


Teacher: ...brines ud another Quality that we will be 
talking about further down the line, the fact 
that some teachers resent ever being wrong. 


3- 

c 
3 


1 


Z 


Student: I think sometimes some of my teachers act 
A: like they are condemning some of my other 
teachers for something my other teachers are 
doing. 


9 


7 


AA 


Student: I don't understand. 
C: 


9 


8 


BB 


Teacher: I do* I think he means that sometimes the 

teacher criticizes one of his other colleagues 
in front of his class. 


3 


3 


CC 


Student: Right* some of my teachers do that. 

D: ; 


9 


7 


DD 


Teacher:" You think that is not cood? 


4 


4 


EE 


Student: Yeah* 'cause it makes us think well there's 
D: a teacher talking about other teachers. 


8- 
9 


7 


PP 


Student: And if they talk about other teachers, what 
C: do they say about us behind our backs? 


9 


7 
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Segment 


(Cont'd) 
Illustration of Categories #1 
High School English Class 


Flanders ' 
Category 


Cognitive 
Category 


GG 


Teacher: Among teachers, they can call that 

professionalism. Their relationship .within 
the profession itself. Would you like. to 
add a category, professionalism to the scale 
we are developing? 


3- 

4 


■ 1- 
2 


HH 


Student: They don't act like teachers. 
A: . 


9 


7 


II 


Teacher: You mean, the.v don't act as you think 
teachers should. 


3 . 


1- 
3 


JJ 


Student: I had teachers who never learned. all .the 
D: students' names during the -year. 


9 


5 


KK 


Teacher: You resent their lack of interest in ynu as? 
an individual. 


1 


9 


LL 


Teacher: I'm afraid I'm orettv bad about that. ln my 
classes, too. 


3- 

5 • 


1 


MM 


Student: Old lady Adams has nets and I don't think 
A: that's fair. 


9 


9 


NN 


Teacher: We must keep this on an anonymous scale 
.because we are not dealing in specific 
personalities. 


7 


9 
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Flanders' 
Cateaory 


Cognitive 
Category 


Segmen 


Illustration of Categories #2 
High School Speech Class 


A 


Teacher: Good morning class. I. want to remind you that 
your speech communication models are due Friday. 
Please do not wait until Thursday night to- start 
them. Put them on my desk the first thing Friday. 


6 


1 


B 


(Three Second Pause) 


10 


10 




Teacher: We were talking about the elements of persuasion 
yesterday when the bell rang. We noted that there 
were three components. 


5 


1 


p 


Teacher: Who remembers what they are? 


4^ 


2 


E 


Student: Ethical proofs emotional proof, and logical 
A: proof. 


8 


5 


F 


Teacher: Good^ now which of these do you think is the 
most important? 


u 


u 




Student: Ethical proof is the most important because 
A: if the speaker doesn't have a good personality, 
or image, or reputation, you won't -listen to him. 


8- 

9 


7 


H 


Student: I would think that the emotional aspect would 
B: be more important, because after all, he's trying 
to get the audience to accept what he's saying. 
He could do it better if he appealed directly 
to them. 


9 


7 


I 


Teacher: In other words, you think that the most . 

powerful tool is in the audience and not in the 
speaker himself. 


3 


3 


J 


Student: Well, yeah, if he can get the audience "with" 
B: him, emotions can sweep people along, and be a 
strong persuasive power. 


9 


7 
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Segment 






(Cont'd) 
Illustration of Categories #2 
High School Speech Class 


Flanders ' 
Cateaorv 


Cognitive 
Category 


K 


student: But you're looking at it wrong. We want to 
C: know what the speaker does, not what the audience 
does. 


9 


5 


L 


Student: It is the sneaker. iiRing- s.„^-j or,^^ i c 
B: emotions. Dummy! 


9 


5 


M 

TVT 

N 


Student: Dummy yourself! T nan t-.hin^ "hat I want' 
C : 


9 


9 


Teacher: It makes you mad ho Hp lahoiori pxpe^lally 
when you are reasoning the facts as you see 
them, and you both are exploring the same 
problem from different angles. 


1 


9- 

3 


0 


Student: Ya know^ I guess t jnsh riiri wha^ an 

C: audience would do if a speaker called them a 
name . 


9 


7 


P 


Teacher: That is a very perceptive statement. Since 
you react in anger to being labeled, you can 
understand how an audience would react if a 
speaker called them a name. 


3 


3 




Student: Well, now I can see whah happens if th- 
B: speaker resorts to name-calling. 


9 


7 


R 


Teacher: This brines up a g;nnri pnini- r.^t's list 

'-'A'^w.'^^i av^i/xuiiD uxiciu diouse auuience emotions* 
We can begin with what just happened here ... 


3 


3 


S 


Teacher: "Antagonizing . aual ence hy laheiing; th°m " 
Any other ideas about what arouses audience 
emotion? 


5- 

4 


1- 

4 


T 


(Three Second Pause) Teacher writes on board. 


10 


10 
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Segment 


(Cont'd) 
Illustration of Categories #2 
High School Speech Class 


Flanders ' 
Cateqory 


Cognitive 
Category 


TT 
U 


Student: I think a speaker would want to make the 
D: audience feel he is "one of them," you can 
trust me because we're alike type of thing. 


9 


7 




Teacher: Great. We sometimes call that "Plain 
folks Appeal." 


2~ 
3 


1 


W 


(Three Second Pause) Teacher writes on board. 






Y 


Student: How about when the speaker makes everything 
E: seem bad on one side and good on the other, and 
we're good, so naturally we all fight the bad. 


9 


7 


Y 


Teacher: Do you mean like saying Gommunis^m is bad, so 
we have to hate the Russians? 


3- 

i\ 


3 


Z 


Student: Yean, somethln' like that. 
C: 


8" 


5 


AA 


Teacher: Since everybody else feels this way, you 
ought to feel this way, too. Sort of "you, 
too. . . " 


3 


. 1 


BB 


(Three Second Pause) Teacher writes on board. 


10 


10 
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Segment 


Illustration of Categories #3 
College Educational Psychology Class 


Flanders ' 
Category 


cognxtxve 
Category | 


A 


Teacher: Good morning. Today I want to continue with our 
discussion of learning theory, so I'd like for you 
to get some paper and take notes. I'll collect 
the notebooks at the end of the week and grade them 
for completeness and order. 


6 


1 


B . 


(Pause) 


LO 


LO 


C 


, Teacher: Yesterday we diocussed the idea that there are 
several different explanations for human learning, 
and yes, it is possible to divide them into two 
general groups. This is something like the 
biological classifications of animals and plants. 


5 


1 


D 


Teacher: Now, who can name the two major families of 
learning theory? 


4 


2 


E . 


Student: The two major families are behavoristtc and 
A: field theory? 


8 


5 


F 


Teacher: Right, Now, which one of the families deals 

primarily with precise factors and which considers 
more general components of learning? Maybe it 
would be clearer to ask which of the two families 
is more scientifically rigorous? 


4 


2 


G 


Student: The behavioristic family tends to be more 
B: . scientifically rigorous, and tends to deal with 
very precise factors of human learning. 


8 


5 


H 


Teacher: Right. What are some reasons why there are 
different kinds of learning theories? That is, 
if we're talking about the way people learn, why 
have we come to different conclusions? 


4 


4 


I 


Student: Well, people don't see things the same way. 

A: It's like the expression, "Beauty is in the eye of 
the beholder." 


8 


7 
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Segment 


(Cont'd) 
Illustration of Categories #3 
College Educational Psychology Class 


Flanders ' 
Category 


If— 

JO 

30) 
^ 


J 


Student: You know I think the real problem is that we're 
C; not scientific enough in our study of * learning. 

What I'd like to see is for someone to try all the 
learning theories on the same group of students and 
find which one explains their learning. * 


Q 


7 


K 


Teacher; If I hear you right, what you'd like to do is to 
test the theories in a "real" situation so you could 
decide which one you think is the right explanation. 


3 


3 


L 


Student: Yeah, surely one of the theories is better than 
C: the others, or at least we should have some clues 
after all the researching we've been doing. 


9 


7 


M 


Student: Hey^ it seems to me you're hung up on the notion 
B: that knowledge must be either right or wrong. You 
seem to like categories rather than dealing in 
shades of gray. 


9 


7 


N 


Teacher: You must see his thinking about learning theory 

as only a part of his more general thought processes 
^which you feel are focused around categories. 


3 


3 


0 


Student: Maybe I do deal in "Black and White" thinking 
C; but putting a label on me is not dealing with the . 
problem of how people learn. 


9 


7 


P 


leacner; xi I'm nou mxsuaKen you leel a llutle upset 

because he avoided your question and tried to make 
your thought processes the problem. I certainly 
can understand your feelings and want you to... 


1 


9 


Q 


Teacher; ...know, that I found your questions very stimulat- 
ing. In fact, the class's enthusiasm has been very 
refreshing to me. I really like your active minds. 
Let's keep it going. 


2 


9 
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Segment 


1 

•Cont'd) 
Illustration of Categories #3 
College Educational Psychology Class 


Flanders* 
Category 


Cognitive 
Ca€egoiY 


p 


Student: Great! I did feel defensive, but I'd like to 
C: suggest seriously that we conduct studies of 

learning theories in this class. I want to try 
them under controlled conditions^ I really want 
to know about them! 


9 


9 


S 


Teacher: Wonderful! You make me feel excited about 
that approach o 


2 


9 


T 


Teacher: Does anyone have specific suggestions about 
how we could do this study? 


4 


4 


U 


Student: I do, but I'm not sure of all the details* We 
could be "Guinea Pigs" for each other and each of 
us could conduct his own experiment and see for 
himself. 


8~ 
9 


7 


V 


Teacher: Good suggestion! We could each take one theory 
and apply it to a learning sicuationo 


2^ 

3 


3 
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APPENDIX B 

Master ratings for the three basic training tapes are displayed 
on the following pages • The masx^er ratings on each sheet are 
arranged sequentially by number as they occur on that tape» 

Some of the segments are repeated on the different training 
tapes. This is to demonstrate behavior correspondence among ratings 
on the seven instruments e Frequently,, a behavior .that is a high or 
low example on one scale is also a high or low example on another* 
The. chart below shows the various locations wf repeated segments* 



un rxanuers ana 
Cognitive Tape 


un 

Meaning Tape 


— ?^ 

un 

Student Involvement 


Segment 1 


Segment 5 


Segment 7 


Segment 2 


Segment 9 


1- 


Segment 3 


Segment 17 


Segment 4 


Segment 4 


Segment 2 




Segment 5 


Segment 11 




Segment 6 ' 




Segment 8 


Segment 7 


Segment l6 




Segment 9 


Segment 6 


Segment 5 


Segment 10 




• 

Segment 9 


Segment 12 


Segment 3 




Segment 13 




Segment 11 


Segment 14 


Segment l4 






Segment 7 


Segment 1 




Segment 8 


Segment 2 




Segment 15 


Segment 10 



80 



I 



H 
H 

& 

H 
M 

i 



U4 

a 
o 

CO 
H 

i 

» 



^uduaAiOAUi 



ssauauxnuao 



aAT:jTu6oo 



a 

0) 



o 
o 



t o 

H 0) 
CO 



-xoadcJv 
aurpx 

t^uamSas 



to 



in 



CO 



in 



in 



in 



% o 



in 



0) 

o 

0) 

• o 

H a 

0) o 

> -p 

0) o 

H a 
o 



6 (Q 

U M 

O 0) 

• 3 ^ 

Bco o 

fH ^ (D 

to O H 



0) 



a 



0) p 
O 



*H o a 

.i^t o 

0) H -P 
04 P4 O 

O (D 04 



(1) 



0) 
O 

c: 
(d 

4J 
(0 
•H 
0) 0) 

(d M 



(d a 

= a 
(d 



pa 



^ > 

3 H 



(1) 

(1) 'd 



0) 

a 
o 

•H 
-P 
0) 

(1) ^ 

D* a) 
a (d 

*H 04 CO 



(D 

-p 

(D 
H 
C 



(d M 
o 

C -P 
o » 

<D S 
CO 



to 

M 



% O 



in 



in 



^ 00 



IM 
O 

(1) 

(d 

(d 

o 
a 
o 
-p 

^ CO 
Wl - 
0) ^ 

o 

^ fd 
o o 



> 
<1) 



fd CO 

-P V 

(d (d 

o u 

O <1) 

u 'd 



(d 



> u -H 

rd -H 

04 (d w 

(0 O4D 



(d ^ 

4J (0 



a 



CM 



to 



in 



in 



in 



H in 



in 



^ \X> CO 



m 

H 0) 

(d a 

o o 

(0 &4 



(0 
CO 

0 
o 
o 
u 
O4 



rd 
o 

B 



O4 
U 

•g 

(d 

<u 

Eh 



CO 

o 
'd 

4J 
(0 

IM 
0 

>1 
-P 
•H 
H 

fd 



rd tn 
-p a 
c -H •d 
o "d ^ 

§rd p 
<i) S 



a 
g 

CO 



CO 



rd 



B-3 



ERiC 



SI 



:*uataoAXOAUi 



-P 
c 
o 



o 

88 



uoTtjoiuoad 



ssauauTnuao 



5uTu^aw 



aATt^TuBoo 



.saapu^Tj 



CO 

c 

Q) 



o 



in 



CM VD 



iH in o 



in 0^ 

^ VD 00 



CO 

CO CO 

CO 0 (3 
rd 



fd H ^ 
c 



4J 
O 

(0 



0) 



O H 



H CO 



0) 

Q) <U U 
> > CO 



CO 

U 
0) 

u 

U 

o 
u 

0) 

•§ 

CO 
M 

2 

M 

0) 

0) 



O 

CO 

0) 

:s 

H 
O 

«H 
O 



CO 



U 
O 

CO 

c 

0) 

4J 
CO 



0) 
CO 

rd 
0) 

•§ 

0) s 



9« 



10 4J 



in 



in 



<N VO 0\ 



H in 



CO in o> 



H ^ 00 



(0 

c 

H 



H C 

C H 
0) 0) 

0) 

0) 



M 

CO ^4 x; 

0) 

CO 4J 

0 o m 

$0 0) 



4J 
t U 
H 0) 
0) -m 

^ CO 



a:^BUiT 
-xojddv 
aurrj, 

:(uaui6as 



u 

rd Q) 

4J u 

o o 



O 0) 
^ CO CO 



to rd 



0) 
H CO 



(d 
c 

u o 

^ tJ^ CO 

•p as H 
'H (d 
^ o 

• CO 
0) CO 

0) CO 

C C CO 
O -H 
ftH 
CO 0) 
0) 0) 



rd 

g rd H 

B (d *H 



o 

CO 
(0 

o 

CO 



c 
6 

OJ 



in 



o 

• 



o 



CM 



o 

CO 



o 

4 

CO 



CM in 

% % % 

H 5t 



^ <n 



CM LO 00 



! 

0) 

> 

0 



s 



0) 
0) c 

§° 

CO 

•d 

• 0) 

0) 4J 
CO 'H 

•H a 
rd -H 

U H 
Oi 0) 

0) 4J 

6 C 
O 0) 

CO 6 



0) 04 

•d 



c 
(d 



4J 
M 

o 

•d 

i (d 

0) 

CO 0) 

c (d 



St; 

o c 
(d 
o 

M 

^ O A 



CO 
0) 

(d 'H 
•H »d 
a 0 

O 4J 
0) CO CO 
CO 



(d 
•d 

G 

o 
o 



1 



6 



VO 



er|c 



8^ 



UOTt^OUIOJd 



00 



in 



CO 



ssauauxnuag 



BuTueaw 



CO 



in 

CO 



CO 
CO 



in 



n 



in 



CO 



in 



o 



CM in 
r-i ^ 



ro in 0> 
CM ^ vo 00 



C 

0) 



43 

I O 

H Q) 

Q) n 
> 

Q) 3 

* 03 



0) i4 

CO 0) 

(D 

o 

o 

tJ\ • 
o 

0) U4 
CO -H 
0 O 
Q) 

CO CO 
'O 
CO C 

•H to 



0) 



CO 

c 

0) 

CO 
CO 



0) 



CO 



0) en 
0) o 



o 
o 

c 



dr- Q) 

(d CO c 

0) ^ (d 

M 0) 

CP TJ 'O 

C H 

CO (d *H 

(d H ^ 



-H 0) 



<^ •fc «k 

Ol 5t Vi> 00 r-4 

« ^ <* -4^ «V 

fH m in CJ\ 



a> 

^ * ^ o 

ro. in 00 1-4 



o 
in 



B^5 



o 
in 



in 



ir 



in 



•» * % 
CM in a\ 



rH ^ sD 00 



in o> 



CM ^ 00 



^ CO CO! 

0) 0) -P 

OS c 

0) 0) 
X i« 

o ;3 

CO 



C 0) 

M > 

3 Q) 



4J H M-l 



(d 
c 



H O (U CO 

rd H o H 

H c <d 

Q) O fd <D g 

g Q) O H -H 

Q) 13 O C 

'O CO < 



pa 



pa ^ 



-xoaddv 

SUIT J, 



c 



CO CO 

H Q) 

fd CO *rH 

O D H 

O Ck 

> cu 

C 0) 

•H tP C 

(d ? 

CO 0) c 

•H > -H 



M VO CO, 

fd 4j 

-H - C 

H CO 0; 

e 0) q 

fd 13 -M 

U4 C CO 

(d 

o H = 

-P (XI C 



CO 

CU CO 



0) 
CO 

fd 
u 



0) 
• 0) 
0) M-l 

c 

o o 



0) 

o 
c 
o 



o 
a 



fd 



c 
o 

-H 

o fd o 

0) -H CO 
C CO 



^ >1 

fd tP 

13 O 

C H 

o o 

O -H 

0) cn 

0) 



C 

a 

CO 



00 



-p 
c 
a> 

CO 



(Q 

c 

.0) 

-P 
CO 



3. 



u ca 

0) 



H rd 
CU 0) 

e B 

o 

o 



^ 0) 



CO 0) (0 

fd -H -H 

H M ^ 

O O -P 



0) 

0) 
> 

+J id 

C (D 
tP 

O -P 

u c: 

0) 

^« 

0) 

H 

fd 0) 
> 

•P 0) 

fd H 



0) 

> 

0) 



0) 



4J 

o 

<D 

CO 

0) 

a; 

M-* O 

O >^ 



fd 0) 

•P o 

c c 

0) 0) 

e -H 

0) o 

H CO 



ERiC 



?3 



B-6 



c 
o 
o 



a 

H 
H 

> 



2; 

a 
o 
o 



CO 

pi: 



Em 

o 

H 

I 

pi; 
« 



q^uamaAXOAui 


• 


CO 
♦ 

CO 


•* 1 
• 

CO 




• 


• 

CO 


to 
• 

CO 


uox:^ouioa<j 


• 


• 

CM 


U 1 

• 

CO 


ssauauxnuao 
























V 
















CO 






















CO 
















i J 






















• 
















* f 






















CO 






















































%^ u r 






CM 5j» 














CM in 














aAT:^Tu6oo 


r- 1 ^ 








% «^ 


in 












% « 

rH ^ 




























- 


























^ w 








CO in 


00 












CO in 


00 












. s japuBX J 




/TV 






CM ^ 


vO 












CM ^ 
















« 








t * 




' CO 


CO 
























«» n 
^ n 








(D • 






(D 












43 


C 










(0 S 




01 


10 


6 >t 




rH 


0 


C 








.c 


;J 


0 










^ n 






(D 


0 .c 




nf 


'CJ 


•iH 






<D 


43 


.0 






IP 






m S 








CQ Oi 


0 


4J 










tj 






















= itf 


4J 




(D 










'CJ 


CO 


• 










S-H 








It 




M 










c 43 






HI 












-p 


•H 


4J 


- IP 


'CJ 


(D 


>« 








0 CO 


Id 


CO 


'CJ 










1— 1 




U 






(D 


.c 




• 






CO • 


& 


C 


u 








fA 


•H 0 


M 


0 


(D 


-H (D 




0 


UH 




CO 




u CO 




0 


0 


tp 


X. 






(D 


iH 




M-l CP 


0 


(tf 


-H 


(D 


'CJ 


• 


(D 43 


0) 


& 




c 


43 




11 


10 10 


ai> 


8* 


•3 


•H 


1— 1 


(D 






u 




Oi Oi 




a 




















1— 1 




0} 


43 


0 


rH 


! (D 


0 


0) 


tp 




nf 










(D 


CO 




^ 




43 






N 


0 0 


43 


M 


c 












01 






Oi (D 






C 


(D 




N 








-H 


g 


a 






tfn m 
U iD 




c 


CO A 


(D 


• 


(D 










CO 


IP 


rH 


— 






\J 


PI fr4 


t 1 
M 


(D 




4» 


Vl 




•0 


*iH 


C 


0 


c 




C 


0) 








r V 
\^ 


0 :3 


Its 


43 


.c 


43 


(tf 


43 


:j 


c 


•H 




0 




•H 


<D 


rH 












(tf 


43 


0 CO 




Of 


43 


IP 


>i 


tr 


CO • 


0 


rH 


<M 




To 


• 




10 


09 


O4 




«B 


CO 


0« 


CO 


0 


rH 


C 


U XJ\ 




0) 




> 




> 




u 


+> 


-H 


^ 




4J 


-H 




C3 




•H 


(D C 




<D 


43 




CI 


43 




0) 10 


c 






0) C 


c 






(D 




N 


Oi-H 




CM 




rH 


c 






•SB 


0) 


•H 


CO 


CO AS 


(D 


•H 








> 




CO 


0 




0) 


> 






'O 


43 






'CJ 


43 






1 


rH 


>s 0 








CO 


43 






nf r-l 


3 




0) 






U 


> 


43 






U (IS 


(D 




43 






43 






43 


Of 


'O 




43 


Itf 


p 


0 


c 


c 


(D (D 


'CJ 


§ 












t* < 


CQ 




•H 


o« 0 


CQ 


Oi 




C 






> 43 


-H 


CQ 






CO 






>i 




































■ ft 


u 


1 


































1 0 


nt 0) 


•H 








i 




























-p 0 


C 








CO 












43 CO 
















C C 


3 


0 




• »CJ -H 


CO 




rH 








C -H 
















(D 0) 




•H 




C rH 


0 












<D rH 














g H 




4i 




0 tr 


u 


0 


N 








g IP 
















0) 0 


0 


OS 




0 c 


U 












0) c 
















•H CO 








0) pa 






Oi 








rH M 
















H 








CQ 














ca 














aurri 


2 Min. 


• 

c 

•H 

6 


3 min* 




0 




CM 
rH 



t^uauiaAXOAUi 




"» 

CM 




in 
• 

CO 


o 
• 

CM 


uoTq.ouioJd 
ssaoons 


• 

CO 


O 
• 

CM 


SSouauTnuay 


in 
• 
















O 
• 














cn 
















CM 














in 
















O 














n 
















• 

CM 




















00 










CM 












































CO 


in 












in 












m. 
C^ 


in 














in 


00 










S jaciTPT J 


>. 




00 


















































A 






















i 








-P 






























•H 












c 


0) 








c 


0) 




CO 












»H 




•H 














-P 










M 








• 








0) 














0) 




0) 










c 






01 


u 












a> 


«H 


C 






a> 














c 


o 


(Q 




O 


o 


•0 




C9 


c 




:3 










Id 


0) 






-H 


o 






Id 


Id 


-p 


o 




C 




0) 


(Q 


>i 


fH 




o 




'O 






CD 


Id 


• 


>*! 


c 


9< 








m 






C 


u 








Cfi 


P 








> 


Id 


0) 






Id 






u 


-p 


















a: 










o 


- 


u 


o 






•-4 








UJ 






ri 


n 






o 


u 








o 






Id 




o 


Id 


-H 




CO 


? 




o 


Id 










(Q 


0) 


o 




O 








c 




c 


o 


cn 


09 




C 


Pi 




cn 




Id 






0) 


Id 


o 






■P 




o 


01 




0) 




fH 


c 




fH 




•rl 




C 


C 


c 


•H 










Oi 


Id 




fH 








o 


0) 


0) 






« 






1 






0) 


g 


Id 










(n 








B 


0) 


0) 




O 




o 




m 


:3 




0) 




m 


C 




f-l 


c 




X 




o 


O 


0) 


■p 


4J 




B 


0) 






s 


0 




« 


cn 


o 


? 




cn 


CO 






C 








































S O 


>1 




























•H 0) 


u 






o 










Id 












Id 


o 
















c 














c 




■p 










c 


•H 












c 


0) 


c 


Id 










0) 


•a 










1-^ CO 


o 


•H 


:3 


M 










i 


Id 












o 


o 


o 


.Q 












0) 












0) 


CO 


CO 












c-l 














CO 






> 






















-xojddv 


3 min. 


• 

c 

•H 

6 

CM 




CO 
c-l 


c-l 



:juejiaAXOAUi 



uoT:^ouiojd 



04 

§ 

H 
S3 
H 

§ 

H 



o 

CO 

§ 

M 

i 



in 



in 

CO 



in 



in 



in 



in 



in 



CM in 

rH 5t VO Ofc 



o 



0) 

> 



4> 

a 
a> 

-n 
CO 



ro m o> 



n ^ flO 



4> 



5 

U 

o 
o 
c 



0) 0) 

4J o 

CO o 

a -p 

o c 

a g 

o o 

o: > 

> 

a: H 



o 
>» 



5 
g 

ns 

o 
o 



i 

' 2 

C -H 
(D M 

rH C 

a> 

O 43 

a> (0 
a> 

C 4> 

o nf 

43 g 

0) * H 
3 43 



^ Q) O 

'O C C 43 

c a> :3 nf 

o *H o 

a a CO ,Q 

0) CO ^ -H 

CO > 



6 

CO 



CM VO 
rH in (Ti 



in Ok 



> 

• 43 

(0 a 
(0 a> 
nf «M 

a nf 



a> 

43 

a 

43 

m 



rH 
rH 

m 

c 

o 

B 

rH 



10 

0) 

g 
o 
nf 

0) 



CO 



0) 

0) 0) a 
> > w 



43 

a 

Q 

o 
a 

a> 

m 
u 
o 
u 
u 
o 

0) 

§ 

(0 



o 
o 
o 



MH 



CP o 

C 43 

a: 10 
nf 0) 

g :3 

rH 

M a 
o 

U4 MH 
O 

m 

43 a> 
•o g 
m 0) 



nf 0) ^ 

43 o cc 

o o u 

g H -H 

0) a cQ 

rH CO ^ 



g 



CM 



in 

CO 



B-9 



in 

CO 



in 

CO 



CO 



CO 



/ 



CM 



CO 



CM 



43 

c 

1? 



SH 10 X 
0) 43 0) 

Oi Oi 
■ 0) o 
O 43 

o 



nf 



43 

tP o 



I 

o 

43 
I 

c 
o 

CO 

n . 

^ CO 

>f o nf 
M nf 0) 
0) 0) 

> 43 -H 



• o 

0) ^ 



rH 
0) 
0) 
MH 

<D 

i- 

CO 



tp 

•iS 

tp g 
c = 



> 
rH 



CO 



a 

•H 
IM 
•H 

Qi a 

CO <D 

o Of a 

}H CO o 



CO 

5 



CO 
0) 

o 



0) 
CO 



nf 

43 

c 

0) 



pa 



CO 
-H 
rH 
0> 
C 
H 



C 
g 
CO 



CO 



er|c 



?6 



B-10 



q.U9uiaAXOAUi 



o 
* 

CO 



CO 



o 

♦ 

in 



o 
* 

CO 



in 



o 
in 



UOTt^OUIOJd 

ssaoong 



in 



in 



ssauauxnuao 



CM 



CO 



in 



Buxireaw 



• 

CM 



CO 



in 



Aao6aq.Ho 
aAxq^TuBoo 



iH in 



a m a 
^ « « « 

rH ^ VO (X) 



Aio6aq.Ho 
, sjrapu^Xdl 



Q) 

> 



CO 



aq^curp 
-xojtddv 
aurpi 

q.uaui6as 



r^3 in CO 



Ok 



in 



ro in <yi 



CI ^ 00 



(0 

i CQ 
III -H 

-P CO Q« 
•H 0) 



0) 


:3 




43 iH 


43 


w o 


43 rd 


C 




43 


•H O 


0) 


0) 


o 


•H -H 




c 


c 


C 




o 


o 


o rd 


O 




g 


o ^* 


O 




1 


0) o 




0) 


0) 


D 0) 






43 


E 




}^ 


o 






Q) 










u 






o 


43 










C Q) 




Q) 




O ^ 






0) 


^KT CO 



rd CQ 

C rH 
O 0> 
O C 

CQ 



g 
CO 



O «M 
O 

0) 

C 0) 
O 43 
43 rd 

u 



•H 0) 

0) 

> o 
0) c 

rH O 
43 
>t O 

0^ c 
M o 

0) g 

o 

o o 
l4 > 



CD 

=J 

0) o 

43 43 
43 C 
-H <D 

:3 

4> 
• CO 

Bo 

rH 43 
CQ 

c 
o 

•H 43 
X C 

Id <n 

0) 4^ 
Q|43 

CQ rd 



43 Q) 

rd 

(U O 4J 
g CQ 

43 0) (Q 
(d Q) 
Ol CQ ^ 

43 

CO rH C 
rH 0) 0) 
rH > fQ 

*M rH 43 
CQ 

0) O 
^ rH 

o 

rd CO 

EH ^ 



g 

•H 
43 



43 Q) 
Id rH 

CO 

•• C 

CQ -rl 

o c 

-rl -H 
43 Id 
CQ rH 
S Of 



C ^« CO 

^ 0) g 

o til fi 

Id 0) 

CO Oi 43 



43 

o 

J3 
43 
CO 



CO 
43 
C 
O 



rH S4 

Q) O 

<D O 
rH rd 
Oi <D 
g 
O 
O 

m 

CO n3 

43 

CO C 

CO 0) 

rd -H 

rH S4 

a o 



CO tyi 

-rl O 

43 tJ) < 
-rl 

o 

CO >i 

CO M 

o> o 

rH > 

CO rd 

•rl 

^ 43 

43 rd 



o > 

rH O 

o 

H 

43 



o 

X 43 

o 



rH O 



CO • 

o • 

o 
> 



rd & 
o 

C 43 
O CO 
O -H 

0) K 
CQ 



Id CD 
43 O 

c c 



o 

CQ 



•H 
g 

CO 



in 



C 

g 



V0 



er|c 



:^uauiaAXOAUi 


o 
• 

CO 


o 
• 

CI 


• 

CI 


. o 

• 

CI 




in 
• 

CI 


o 

• 


in 
• 

CI 


o 

• 


uoTq.ouioJd 
ssaoons 


CO 

• 

CI 


in 
• 


in 
• 

CM 


o 

• 


ssauauxnuao 


tn 
• 












* • 














in 
• 










O 
• 










CI 


























IN 




















ouTireaH 


If) 
« 












CO 
• 














CO 










o 




































tN 






































































CI 












C| 














IN 


1 

rn 








in 










aATq.Tu6oD 


i-i 


in 












m 












■ 


in 


























































00 










in 


CD 










in 














in 






















vO 














00 












00 








in 


















CQ 
-P 


















* 
















fO 
VJ 










0) 


• 








CQ 




-P 
















o 








•n 










c 


-p 


Q) 






Q) 




•H 






CQ 


I-I 




0) 


M 


a 


CO 






0) 

w 










o 


o 


g 


CP 












P 




•n 






m 

w 






Q) 


• 












a> 


Q) 


c 




M 


CQ 




>l 


O 


43 


43 




CO 


43 


CO 


U 


43 


CQ 








u 


o 




M 


•rl 




Q) 






1—1 






CO 




H 


■ « 
4J 




w 


43 












o 


c 






•s 




CO 


H 


CQ 


iH 




(U 


o 






CO 


tti 


•> 


Q) 


41 


w 


nt 

lO 








9 


lO 


^ 


-P 




(D 


Q) 


•H 


CO 




T» 


tC 






rt. 

KM 






rH 


•d 


n. 

V-M 








g 






0 








(u 


O 


1 a 














\j 




HI 


JJ 




-p 


iH 




Q) 


Q) 








u 


t ■ 


* 




4* 




■ 


fit 
ID 


M 
lO 




Vk 


1 


43 




VJ 




C 


Q) 


1— 1 


1— 1 


M 








CQ 


wm 






m 
Ui 


irt 






rf 




•d 




CO 




n 




Q) 

g 


> 


lO 


-P 




-p 




o 


Q) 


m 










m 

w 


yj 


Ui 
w 












Q) 


o 


-P 


-P 


? 




1— 1 




-p 


CO 






U 


fix 


«-« 










M 




r^ 






1— 1 


o 


•rl 


Q) 


fQ 










ri 








,__| 


w 


jj 


ri 

w 


CD 




O 


0) 

w 


CO 


• 


o 




> 




Q) 






CQ 




c 


u 


43 


o« 


o 




43 


•H 


CO 


M4 


« 


«H 


,c 






o 








g 






10 






CO 


O 


>1 


M4 






43 


M 








O 






^ 








Q) 




•C 




CQ 


CQ 


Q) 


CO 






CQ 


0) 


43 


S4 




^•d 


CO 








M 


• 


• 










& 


•H 




Oi 


13 


Q) 






Ol 


0) 


O 




M 


Q) 


c 


CO 




Q) 


u 




-P 


■g 






M 






CQ 


O 


CO 




M 


<D 


•d 






CO 


43 


•H 


u 




C 


Q) 


Q) 


c 


10 




Q) 


Q) 




Q) 




4JL 


C 


c 




M 




0) 










0) 




Q) 


•C 


;j 


Q) 


Q) 


• 




C 


•d 


CQ 






o 




•g 






.CO 




0) 


& 


CO 


•d 




O 


•0 




-P 


Q) 


O 


Q) 


o 








a 






0) 


O 


•H 




M 




•H 








Itf 


o 


S3 






lO 




o 






o 


CO 


B 


CO 


> 


•rl 


10 




10 


Q) 








5 


Q) 


3 


-P 






Q) 


U4 




CQ 


5 


•H 


0) 




0) 


CO 




V4 




> -H 


> 




rH 




1^ 


-P 


m 


CO. 


O 


i 


t< 


O 




•H 




CO 


M 








i 


0 




[ 'd 










>i 












































-p 


M 












>i 


































i o 


10 












M 














CO 


•g 








ct 








r-l Q) 


-P 












10 














43 










4J 


a 








Q) -r 


c 


O 
























a 


•rl 


















s o 




•H 










•§ 
















rH 








a 


•d 








0) ^ 


i 


CO 










O 














i 












; (0 








»J CO 




:3 










o 


10 
































i-i 












0) 


s 






















1- 
























CO 
























w 










• 

•rl 
g 

CI 


o 

CO 


• 

u 

0) 








• 

•H 
g 

CO 


o 

CO 


• 

o 

0) 

(0 










• 

•H 
g 

CI 


2 min. 
30 


• 

a 

0) 

m 










00 


0^ 


o 

rH 



ERIC 



B-12 



r^uauiaAXOAUi 


CO 


in 
• 

CO 


in 

eg 


0 

eg 




0 

CO 


in 

CO 


in 
* 

eg 


o 
• 

eg 






J 




uoTt^omojd 
ssaoong 


in 

♦ 

CO 


• 


o 
eg 


O 

• 

eg 












ssauauxnuao 


in 
• 












in 








CO 

* 














o 














# 












m 








OI 














eg 


































o 














Q 
























m 








CM 














eg 
































































in 


















OI 


in 












eg 












aAxq^TuBoo 


_J 












1 

rH 


in 






rH 




vO 










rH 


in 
























% 


































'J 


in 


c\ 
















in 


CO 












in 


00 




























CX) 






























1 

rH 




00 








OJ 


in 






vO 


<J> 












vO 


































JO 






























! 


















•p 






(I) 






1 














1 


• 


B 










d) 






05 




i 


c 








5 

w 




(0 




















• 




(0 






Xi 


X 


0) 


o 






















Q) 




rH 




CQ 




•H 








rH 


rH 


-p 




« 


*r| 


CQ 




05 


c 






CO 




0) 




H 




0) 


rH 


05 








05 


0) 






c 




)^ 


> 


H 


05 






< . 


c 


«M 


d) 


CO 






05 










-P 




o 








<D 


5 


?• 






/l\ 

Q) 


0) 


Q) 








O 


-P 


Oi 




0 






U 






tJ 


(D 


> 


\J 


0) 






'O 
















• 






O 


•H 


-p 




1 


V/ 






0) 


C 






r* 






O 




o 


0 




-p 






4J 




(0 


u 


0) 






H 


O 






iD 


(U 


4J 


■P 








-P 


o 


c 




o 




CO 


-H 


05 


c 




05 


CQ 


O 


-P 




(fi 












* 








d) 


o 


-H 


,__ 

a> 




C 


?t 


-H 




V 


U 












(0 






CQ 


c 


-P 




g 




-P 


c 


-P 


O 


O 


^3 


05 












C 






4J 




1 


d) 


o 


c 






0) 


O 




05 


CP 


-P 


O 


(D 


05 








0) 


UJ 




0* 






rH 








CI 


d) 


g 


o 




05 




oJ 






-P 


(0 






0) 


o 


;3 


(0 




05 


-P 






05 


H 


O 


tp 






(0 


g 


u 


u 




-p 








(0 


c 


U 


C 




O 


c 






u 


Oi 


u 








•CJ 




0) 


05 




c 






0 






U 


0 




to 


0) 




c 


0 


Qi 


0) 






c 






d) 


(0 


0) 






o 




•& 


0) 


a 






+3 


(0 


o 


0 












0) 




CO 


Pi 


C 








u 




4J 


(0 


tP 


to 


Hp 




(0 


o 




(0 










c 






o 


?f 


0) 






0) 




C 




C 


to 


05 


to 




c 


0) 


(0 


0) 


4 






o 






-H 


-P 










o 


-H 


u 


•H 


0) 


•d 


s 


+3 


0) 




0) 






• 


0) 


-p 




-P 


(0 


o 


(0 




o 








rH 


o 




0 






O 






-P 


o 


o 




0) 


(0 


05 


TJ 








Q) 


CP 


0) 


0 


o 


rH 


o 




O 


O 


H 




c 




c 


> 


CO 


0) 


u 


0) 


VI 




Q) 


H 


A3 


c 


Q) 


u 


o 


rH 




C 


o 




H 




05 




o 


5 


<D 




0) 


-P 


O 






05 


CO 




%\ 


a 


o 




•3 


05 




CO 


05 




> 








C 






? 
















>1 






































tP 




0 






u 








>i 














u 












I O 






-H 










C 




u 














05 


tP 










H 0) 




C 




to 






-p 




-H 




05 
















C 










0) 




•H 


g 


(0 






c 




+3 




















-H 










0) :3 


0) 




rH 








0) 




C 




C 














0) 


TJ 










g 


(d 


•H 


o 








+3 


:3 




O 


§ 












g 


05 












0) 


0) 




(0 








05 


0 




O 














0) 


0) 










H 






-H 






H 


s 


u 




0) 


1 












H 




















a 






H 








cn 
























at:^Burr 
-xoaddv 
aurrj, 


• 

c 

g 
CM 


2 min. 


o 

CO 






4 min. 


2 min. 


tr^uauiBas 


rH 
rH 


eg 

rH 


CO 
rH 


H 



rmauiaAXOAui 












in 
• 










CO 
• 


















CO 


















in 

9 










in 
• 










in 
• 




























H 






uOTt^ouioaa 


to 










CO 










in 






ssaoons 


• 










• 


















in 










in 










r> 






ssauauxnuao 










































( 














in 










CO 

• 










in 
• 






CM 




















rH 




































CM 










CM 


in a\ 




* 




CM 0> 






9ATq.Tu6oo 


H 


in 


















rH in 










VD 










in r-' 






in r> 






, SaapueX j[ 
























m 

uu 




CM 


in 


CO 






CM 


^ VO 


















• 
















rn 
UJ 




















u 




1 
i 






















to 


0) 'O 








_J /W 










-P 








•H 






uJ 




lu 




rn 

UJ 








0 


-p 


* 


(o 






1 1 




U 4-' 


n 






a) 




to 


-P 


H 


P 




/>~* 1 




rn rn 
UJ UJ 


r« 








H 




(u 


c 


■ Ul 


to 




5 






\J 


rn 
UJ 




0 


f \ 


■p 


rH 


/It 




0) -P 




U4 U 


M 


rn 

UJ *fn 




43 • 


(Q 


H 


c 






i 




tr: c 


o 




rn 
UJ 






-P 




o 


/It 


•s 




O 




c: 


43 C 


i 1 


UJ 




0) 0) 


C 




o 


0 


-P 


/It 


o 




0) 


0 0) 






• 


CO P 


0) 


0 


*H 


•H 


to 




• p 


-H 




It t 


0 ^ 


CP 


P 'O 








o 


? 






o +> 


43 




o 


M to 


C 43 ;Q 


g 


0) 


• 








to 


to 












to p 


0 


H 


to 




-P 


o 


/II 




P4 C to 


1 1 

4-' 


c: 


CP 


to 


o 


GU to 


** 






to 


•H to 




•H U 




H P 


(IS 


MH 




0) 


CP 


/II 


t t 

-P 




O H 


43 


U P 




rH P( M 


0 - 




1 


c 










0) ;c 




P o 




(IS 


p 


0) 






0) 








H 




<D 


43 0 


UJ 




0 


>i c: 










0) 




0) 


to CO 




O 0 


u 


c: <D 


0 


43 o 










rn 

UJ 








tP 


P 


w 


O A 




•H 43 






?J 


0) 




43 


0 


to c 




u ^ 




0 


0) 


rH O 




0 


G 


<1> 


4-* 


/W 

lU 






to 


43 


UJ 






(IS c: 




O 


0) 


-p 




t t 


0) 






to c 




O 0) 


m 


P 0 




o 


o 






CO 


B -H to ^ 








0 


























>1 




























u 






1 0 














H 0 








(d CP 






H d) 












+3 


(fl -H 








43 C 






0) -n 




•H 










•H 43 








C H 






> A 




13 








0) 


O as 








0) 






0) 


i 


0) 








§ 


<U 0 








i (tf 






J— 1 UJ 




0) 


























H 










H 


CO 'O 








H 




















« 








pa 








• 










• 
















-xojddv 


mi 










mi 










mi 








CM 




















ro 






' :vuaui6as 


in 










VD 




































rH 







r^uauieAXOAui 



uoTq.ouio:cd 
ssaoons 



sseueuxnuaa 



BuTUBaw 



aA'cq.Tu6oo 



-P 

0) 



o 
a 



J 



i o 

H 0) 

a) n 

> 



-xojddv 
:^u3ui53S 



o 

CO 



in 



CO 
CM 



in 



in 

CM 



CM 



in 



in CO 



o 



0) 

> 

0) 
H 

>i 
D> 
U 
0) 



0) 

§ 

-p 
o 
c 
o 

0) 

o 



CQ 
-P 

0) 

0) 



0^ 



U TJ 
0). 0) 

0) :3 
<P CQ 



& O 

0) C 

^4. 
-P :3 
0) ^ 
0) 

g M 
0) 

CQ ^ 
-P O 

0) 0) 
TJ -P 

cn o 



M 
-P 

a> *H 

g CQ 

H a 



CM 



o 

CM 



o 



in 



CO 



r 



CO 



CM <ri 
rH in 



in 

^ VP OP 



M 
0) 

0) 0) 

o ^ 



CQ 
(d CQ 



4J 

0) 

00 

as 
U 
Q* 

4J 
O 

c 



-p = 

O cw 

CQ 'd 



o 

to 

CQ -H 
0) 

> = 

O 
O 

0) 0) C!7 
EH M = 



O 
-P 

M 
O 
•H 
CQ 

O 
>i 

0) 

u 

CQ r 



0) 

u 

M 
O 

0) 
CQ 

a 
o 

04 
CQ 
0) 
M 



-P 
C 
0) 

T3 rd 



:3 
■P 
CO 



>1 
VI 

fd 

o 
o 

0) 
CO 



g o 

O 0) 
CO CO to 



CM 



CO 



in 

CO 



CO 



in 

CO 



in 

CO 



CM CTi 
rH in 



CO in CO 



CM ^ VP 



Id 4J 



CQ 
0) 

1 



g 



o _ 

4J > 

O H 

g P 



P CQ 
2 

: Ti 
-P 
CQ O 
H 0) 

rd 0) -H 

CO 



Oi H 

CQ 4J 

CQ C 

0) 0) 

O T3 

CO CO 



CQ 

o\ 

0) 

o 
o 
rd 



a 

•H 
H 
H U 

rd 0) 
o ? 

= CQ 

c 

0) rd 
u 



rd 



CQ 



CQ 0) 



rd 
U 



04 CQ 



0) 

o 

rd H 
0) a 0) 

Eh rd ,C 



VO CQ 

- C 

CQ 0) 

M T> 

<D :3 

T3 -P 

C CQ 

rd 

a 



-p 

CQ 

rd 



U 
rd 

-P ^ MH 

C -P cid* CQ 

0) H rd TJ 

§rd 0) O 

0). M o 

rH ffi PEJ pt| 



e o 

O 0) 
rH CO CQ 



CO 



B-15 



C 



er|c 



B-16^ 



^uauiaAXOAui 



O 



< 

H 



22; 



> 
o 



CO 

o 

CO 



UOT^OUIOij 

ssaoons 



ssauauTuuac) 



BuTu^aW 



aAT^TuBoo 



AjoBaq.eo 



o 



in 



in 



f-i in 



O 
u 



I o 

H Q) 

Q) -n 

> ^ 

Q) :3 

1^ CO 



aq.^uiT 
aurri 
r^uauiBas 



in 



^ VO 00 



Q) H 
H H 

0) 

w > 
CO 

0) CO 

O 0) 

O X 

^ CO 

C 

H ?J 

C 0) 
O X 

o 

O 0) 



4J 
C 
0) 

CO 

M-i ;3 
O CO 

•P 0) 

•rl C 

H O 

?J o 

O 

Q) ^ 
C 
O 



C CO 

Q) 'O 

g M 

0) O 

Da 



•rl 
g 

CO 



o 

* 

in 



o 
in 



in 



in 



in 



CM in (Tk 
H ^ vo 00* 



CO 
CO 



in 

CO 



in 

CO 



in 

CO 



CO 
CO 



CO in <y> 



CM ^ 00 



u 

0) 

o 
fd 

0) 

H 

Q) ♦ 

0) 0) 
+J 
C 
0) 
•rl 

O 



0) 
H 



I 

O 
CO 



C 

CO 0) 
CO 'O 



H 



;3 
-p 

CO 



O tT> 
CO -H 
CO X 



CO 0) 

Q) > 

fd fd 

H 

0) ^ 

M fd 



0) ^ 



o 

0) 
Oi 0) 
CO ' 
0) 



0) a: 
> 

0) M-I 

H O 



fd 0) 



<D 0) 

g 

0) o 

H CO 



•rl 
g 



in 



CM in 
^ « « 

H ^ 



CO in (T> 
* * ^ 

CM ^ 00 



CO 

fd 

0) 

CO -rl 

o c 

^ 0) 

:3 



o 

CO 4J 

c o 

H ^ 
0) 
0) 



0) 
CO 
•rl 

fd 

M CQ 

CO 0) 
0) O 
CO O 



•rl 

O 0 

-p ^ 

CO CO 

•0 -P 

c c. 

O 0) 

CO p 

0) -P 

02 CO 



CO 

O -rl 

tn 0) 
c > 



fd 

0) 

0) CO 

tn 0) 

•rl p 

CO 4J 

;3 CO 



0) 

> 

•rl 

fd 

•rl 
•rl 
•rl 



M CO 

fd H 0) 

♦a fd -rl 

C -rl 

O O -P 

0) CO CO 
CO 



•rl 
g 

CM 



VO 



ERiC 



:jU9UiaAXOAui| 


o 
















O 










• 












:^u^pn:^S 


















* 

CO 










CO 














in 
« 
















o 

• 










• 












H 
















CO 






















UOTt^OUlOJd 




























• 












ssaoong 


H 
















> 

eg 










CO 














CO 
















o 






















SS9U9UXnU9D 


• 
















• 










• 












H 
















CO 






















6UTUB9K 


* 
















o 
• 










CO 
• 














_j 
















\ 1 










CO 






















































VO 


















in 








CN 


in 










9at::^tu6od 


H 


o 

H 














H 




% 






H 


% 


% 






































CD 














o 








































r-f 














eg 


in 


CD 






CN 


in 






















Q) 






1 

Q) 




c 

o 










Q) 




C 
















H 






> 












CO 




•H 
















Q< 




CO 


H 


1 




0) 






-P 














CO 


H 


4J 


rd 




o 


Q) 


0 


C 




CO 






C 


rd 












H 




C 










> 


o 




fd 






Q) 






•H 








r-4 


> 


Q) 




CO 


Q) 


o 


C 


o 










^0 


CO 




? 








•H 




'0 






H 




•H 




CO 








:3 




Q) 












H 






C 


B 


0 




0) 




U 




H 


+j 


C 


+J 


CO 




(Q 


o 






4J 




•H 


4J 


Q) 




Q) 


Q) 




H 


CO 


Q) 


rd 


rd 






u 










§ 




g 


o 


'0 






rd 






Qi 


Q) 




4J 


H 


Q) 


5 


c 


C 




C 


5 




C 


O 






CO 




•H 


n3 




c 






H 






•H 


Q) 


0 


cn 




rd 






CO 






O 


•H 






> 






• 


5 


rd 




-H 




♦ 


H 






-P 


D 


CO 


•H 












CO 


Q) 




H 




+J 














H 




+J 




f 

rd 


g 


H 




•H 


>-t 


CO 




o 


C 


• 


0) 




o 


• 


Q) 


H 




U 


3) 


O 








•H 




>? 


o 


0) 


Q) 


4J 








O 




• 


rd 


c 




u 








4J 


Q) 




CD 


4J 


to 


•H 


c 




•H 


o 




CO 




cn 


















4J 


•H 


o 


0 






rd 




CO 




rd 






• 






c 






(d 


fd 


•H 


•H 


• 






* 




0 


c 






Q) 






o 


(d 




- 


u 


r-4 


4J 


•P 


o 




CO 


o 


-P 




+J 




C 




? 




-p 




* 


Q) 




Q) 


rd 


c 


iP 


Q) 




CO 




u 


o 




Q) 








CO 




CO 


H 




-P 


0) 






Q) 


•H 


U 


0 


Q) 
















-P 


Q) 


4-> 




1 




o 


? 




Q) 


H 






•> 






(d 


CO 




g 


4J 




C 


o 






rd 


CO 




cn 


P4 


n 




5 






CO 


Q) 


fd 


Q) 


•H 


5 


Q) 




Q) 


rd 


Q) 


c 


o 


rd 




3 












u 






H 


CO 


6 




CO 


H 


E-i 


rd 




Q) 


Q) 


CO 






























>i 






























>i 










u 




1 








i O 


















U 




CO 






rd 


Q) 


•H 








H Q) 


rd 
















(d 


1-4 


Q) 






+J 


o 


c 








Q) -r- 




o 














'0 


(d 


•H 






C 


C 


:3 








*> o 

»^ fM 


c 


•p 














G 


•H 


13 






Q) 


Q) 




o 






Q) 0 


o 


to 










• 




O 


o 


:3 






1 


•H 










f-^ CO 


o 


•H 














U 


o 










O 


o 










Q) 
















Q) 


CO 


CO 






H 


CO 


u 


rd 








cn 














CO 
















o 






9q.BUIT 


• 

c 
















• 










• 












-xojddv 


•H 

e 
















•H 

e 










•H 

6 












9UIXi 


CO 


























eg 












:^U9Ux59S 




CO 





9^ 



B-18 



a 
o 

o 



§ 

H 
H 

Eh 
2 



O 



Eh 
53 
H 

S 

Eh 
CO 

02 

o 

CO 

o 

53 
H 

I 

02 
EH 



UoUZo A X O A U X 


• 




















CM 








• 












in 
« 








in 
« 












CM 








CO 










uoxr^ouiojcd 


in 
• 


















sseoons 










• 

CO 










ssauautnueg 


in 
• 








in 
« 
































in 
• 








in 
• 












CM 








CO 












OJ 0> 








CM 










aAxr^TuBoo 




















H in 








H 


ro in 










»* % 

^ vO 








CO 


in o> 












CX) 
















(M If) 






r\i 






















rn /It 


























> 












UJ (Q 










4J 




:p 






P M 


'D CO 








CO 
























> 
i> 






M CO 


fO 


• 












CO 






CO 




CO 


0 +J 




4J 






UJ *H 


C 




H G 


d) 


Pi 


r! 


a 


o g 




0 






0 


fU 

w 


w 








•H 0 








ri 
\j 












-P O 




(1) 


0 


*H 


& 


1 1 




EH 




CO tJI 






d) • 


0 


0) 


r! 




0 


r 1 M 
U fO 


0) 




E 




CO 


At 












0 


H CO 






P 




• 




CO 




U 


Qa CO 






S 




'0 CO 




fd 


t 




S (U 


tn 




4J 










CO 






C 


o 


rrt 


c 


-p M 


CO CO 


CO CO 


a 




X 0) 






0) 


fd o 




-P -P 


o 






CP 


CO 


H 






c c 


•H 




•H 


O 






H 


•^'^ 


•H O 


0) (U 


4J 






rd 


0 


4J 


0) 


•is 


H Dj TJ 


CO 






0) 


u 


CO 


O 




0) CO 




0) 




0 0) 






/mi 

lU 






0) 0) 


-p -p 






a CP 






rH 




CO g 




CO CO 






>* 


















4J 


u 


















1 O 












0) 








H d) 


■p a 










o 




















a 








^ 'S 










c 


0) 








0) p 


e 








0 










CO 


0) 0) 








o 


o 










H 02 








0) 


CO 










H 








CO 










• 








• 










-xojddv 


TUI 








TUI 










9UITJ, 


CM 








CO 












O 
H 


H 
H 



er|c 



C-1 



APPENDIX C 



, The three basic tapes used in the rater training are: 

1. Flanders' and Cognitive Training Tape 

2. Meaning Training Tape 

3. Student** Involvement Training Tape ^ 



Copies of these training tapes are located at: 

Juvenile Problems Research Section 

Applied Research Branch 

National Institute of Mental Health 

5600 Fishers Lane 

Rockville, MA 20852 



National Consortium for Education 
P. 0. Box 64952 
Dallas, TX 75206 
Telephone: (214) 692^9726 



Mrs. Martha Willson 
Northeast Louisiana University 
College of Education 
Monroe, LA 71201 
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